There is a dynamic-linking-conflict between different libjpeg dynamic libraries on OSX. First there is a standard native libJPEG.dylib (in /System/Library/Frameworks/ImageIO.framework/Versions/A/Resources/). But if you are using MacPorts, you can also have a port-related libjpeg.dylib in (in /opt/local/lib). The latter may for example have been installed as a dependency for some other port.
This creates a problem when you link against your system libJPEG (which is preferred).
Then if /opt/local/lib is in DYLD_LIBRARY_PATH, that path will be prioritised when searching for a dynamic lib, resulting in a runtime error when loading symbols:
dyld: Symbol not found: __cg_jpeg_resync_to_restart
Referenced from:
/System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
Expected in: /opt/local/lib/libJPEG.dylib
in /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
Trace/BPT trap: 5
So I have two questions (likely related):
What is a good way of solving the actual problem (removing /opt/local/lib from DYLD_LIBRARY_PATH obviously solves it but creates problems for other dependencies)?
What other paths are searched for dynamic libs (I.e. Where is the "/System/Library" path specified) and why does DYLD_LIBRARY_PATH rank higher priority-wise?
I experienced similar problem while using OpenCV in MacOS El Capitan. Solved the problem using the solution in the link
Solution is to delete some dlylibs in the /usr/local/lib directory and create symbolic links to related files /System/Library/Frameworks/ImageIO.framework/Resources/
cd /usr/local/lib
rm libgif.dylib
ln -s /System/Library/Frameworks/ImageIO.framework/Resources/libGIF.dylib libGIF.dylib
rm libjpeg.dylib
ln -s /System/Library/Frameworks/ImageIO.framework/Resources/libJPEG.dylib libJPEG.dylib
rm libtiff.dylib
ln -s /System/Library/Frameworks/ImageIO.framework/Resources/libTIFF.dylib libTIFF.dylib
rm libpng.dylib
ln -s /System/Library/Frameworks/ImageIO.framework/Resources/libPng.dylib libPng.dylib
You should not set library paths using DYLD_LIBRARY_PATH. As you've discovered, that tends to explode. Executables and libraries should have their library requirements built into them at link time. Use otool -L to find out what the file is looking for:
$ otool -L /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
/System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO:
/System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO (compatibility version 1.0.0, current version 1.0.0)
...
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1197.1.1)
For an example of one of my homebrew-built programs:
$ otool -L /usr/local/bin/gifcolor
/usr/local/bin/gifcolor:
/usr/local/Cellar/giflib/4.1.6/lib/libgif.4.dylib (compatibility version 6.0.0, current version 6.6.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 159.1.0)
Note that it references /usr/local. If you've built it in such a way that it references the wrong library, I recommend rebuilding and pointing it to the correctly library.
If that's impossible, it is possible to edit what path is used using install_name_tool, but there are cases where this doesn't work, such as if the new path is longer than the old path and you didn't link it with -header_pad_max_install_names. Rebuilding with the correct path is preferred.
Note that there are a few "special" paths available that allow libraries to be found relative to their loader. See #executable_path/ and its kin in the dyld(1) man page.
If using Qt Creator, you have to uncheck the Add build library search path to DYLD_LIBRARY_PATH and DYLD_FRAMEWORK_PATH option from the Run section in the Projects tab:
I had a similar error when trying to run Apache Celix on macOS Sierra
If you use Homebrew to install libjpeg, libtiff, libpng which may confuse the linker to use macOS imageIO library. Simple fix is unlink those libs:
brew unlink libpng
brew unlink libtiff
brew unlink libjpeg
Re-link those libs whenever we need to:
brew link libpng
brew link libtiff
brew link libjpeg
I had a similar error, and i solved putting the following variable in my bash_profile:
export DYLD_LIBRARY_PATH=/usr/lib/:$DYLD_LIBRARY_PATH
I followed the instructions mdemirst suggested and it fixed my issue. I am using OS X Sierra.
I created a gist just in case someone else runs into the same issue.
Gist to fix Spidermonkey errors
Related
I am creating a C library which is to be built with cmake, using Mac OS for development. In the CMakeList.txt, I have the following
#htslib
find_package(htslib REQUIRED)
include_directories(${HTSLIB_INCLUDE_DIR})
target_link_libraries(projectname ${HTSlib_LIBRARIES})
which outputs upon cmake ..
Found hstlib
However, upon make, I'm getting linker errors:
clang: error: linker command failed with exit code 1 (use -v to see invocation)
So...it can find the library, and the library is definitely installed with sudo make install, but there are linking errors only with this library.
(1) I'm guessing that find_package(htslib REQUIRED) is finding something else. How do I find out what?
(2) How do I explicitly write in CMakeList.txt to find the library which I know has been installed correctly?
Use VERBOSE=1 make to see the linker output. Search for -lhtslib
Read the documentation for the specific Find<LIB>.cmake.
Your specific questions:
"How do I find what CMake found": Use cmake-gui or ccmake. They both show the same info, but one is a GUI and the other is a Curses interface. In the advanced mode ("t" on ccmake) you will find all the variables for the searched packages. Additionally, you may use MESSAGE(STATUS "Found htslib at: ${htslib_LIBRARIES}").
"How to explicitly write in CMakeLists.txt where the library is?" Please, do not do that! CMake is meant for abstracting exactly this kind of information away. You have two options, first the good one: configure cmake on the command line (or in the GUIs mentioned above) to get a CMAKE_MODULES_PATH or a more specific hint to the library -D htslib_PATH=/usr/local/.../ (pointing to the dir where libhts.dylib resides). The worse solution would be to provide a HINT to find_package. find_package(htslib REQUIRED PATH /usr/local/lib) or find_package(htslib REQUIRED HINT /usr/local/lib /some/second/path/where/it/may/be).
Solution
Your linked project has a custom FindHTSlib.cmake link. This one uses pkg_config to configure the library. To replicate your problem, I used brew to install htslib. The pkg-config file can be found (for me, but brew info htslib tells you) under /usr/local/Cellar/htslib/1.8/lib/htslib.pc. So, let's give CMake the required hint.
I couldn't test this, because for me it found the htslib package directly with no further hints.
git clone https://github.com/D-Lo/bamdb # I am using version f5f03d0
mkdir -p bamdb/build; cd bamdb/build
brew install ck htslib lmdb
cmake .. # -G Ninja recommended, but needs brew install ninja
make # lot's of missing symbols
I would recommend to change in CMakeLists.txt the minimum required version of CMake from 2.8 to 3.10 (or at least 3.6).
This is the error I get:
[ 62%] Linking C shared library libbamdb.dylib
/usr/local/Cellar/cmake/3.11.1/bin/cmake -E cmake_link_script CMakeFiles/libbamdb.dir/link.txt --verbose=1
/Library/Developer/CommandLineTools/usr/bin/cc -Wall -g -std=gnu99 -fPIC -dynamiclib -Wl,-headerpad_max_install_names -o libbamdb.dylib -install_name #rpath/libbamdb.dylib CMakeFiles/libbamdb.dir/src/bam_api.c.o CMakeFiles/libbamdb.dir/src/bam_lmdb.c.o CMakeFiles/libbamdb.dir/src/bamdb.c.o
Undefined symbols for architecture x86_64:
"_bam_destroy1", referenced from:
_get_bam_row in bam_api.c.o
_deserialize_func in bam_lmdb.c.o
This can be fixed by adding the following line in the CMakeLists.txt, after the line add_library(libbamdb ${SOURCES}):
target_link_libraries(libbamdb ${LIBS})
Some further notes: you now have a library with a main function. This is because ${SOURCES} is used to build the executable and the library. That can have unexpected side effects. Unless it's needed, don't do it.
I'm having a rather strange issue while building a C++ project on OSX using CMake, while pulling in libpng as a dependency. I have libpng 1.6.21 installed via homebrew and the following CMake rules:
FIND_PACKAGE(PNG REQUIRED)
INCLUDE_DIRECTORIES(${PNG_INCLUDE_DIRS})
LINK_DIRECTORIES(${PNG_LIBRARY_DIRS})
ADD_DEFINITIONS(${PNG_DEFINITIONS})
When CMake starts to build and finds the dependencies, it outputs:
-- Found PNG: /usr/local/lib/libpng.dylib (found version "1.4.12")
Investigating further, /usr/local/lib/libpng.dylib is a symlink to brew's 1.6 version:
$ ls -l /usr/local/lib/libpng.dylib
lrwxr-xr-x 1 fluffy admin 40 Apr 9 16:06 /usr/local/lib/libpng.dylib -> ../Cellar/libpng/1.6.21/lib/libpng.dylib
However, it appears that it is the incorrect png.h that is being included, as printing out PNG_LIBPNG_VER_STRING at startup outputs 1.4.12. And, of course, when I try running my program, I get a version mismatch and the library fails to work:
libpng warning: Application built with libpng-1.4.12 but running with 1.6.21
libc++abi.dylib: terminating with uncaught exception of type std::runtime_error: [write_png_file] png_create_write_struct failed
Using FIND_PACKAGE(PNG), the -I declarations never appear in my build line when I build with VERBOSE=1. However, if I use the PkgConfig approach:
FIND_PACKAGE(PkgConfig)
PKG_CHECK_MODULES(LIBPNG libpng16 REQUIRED)
INCLUDE_DIRECTORIES(${LIBPNG_INCLUDE_DIRS})
LINK_DIRECTORIES(${LIBPNG_LIBRARY_DIRS})
LINK_LIBRARIES(${LIBPNG_LIBRARIES})
ADD_DEFINITIONS(${LIBPNG_DEFINITIONS})
the correct -I flag does appear, and yet it's still using the system png.h instead of Homebrew's.
Is there any way to force the compiler to use homebrew's png.h? I can't simply uninstall the homebrew libpng since some of my other packages depend on it, including other libraries that this program makes use of.
EDIT: As a temporary workaround I've just added /usr/local/include to my INCLUDE_DIRS() and included libpng16/png.h instead, but this is a fragile hack.
Have stumbled on this infuriating bug today and spent some time to get to the end of it.
The problem is that classical cmake-style Find*.cmake search for the headers and libraries separately - and the results MAY and WILL mismatch in some cases.
MacOS exaggerates the problem by having special case of frameworks, being searched BEFORE other locations by default.
In my case cmake finds headers from /Library/Frameworks/Mono.framework, which of course are outdated and doesn't have the libraries at all.
You have the options of:
set(CMAKE_FIND_FRAMEWORK LAST)
This fixes the issue with the rogue frameworks like this(in my case)
This is the fast dirty fix.
Use PackageConfig as you originally did - This is the recommended long-term solution.
Using PackageConfig prevents lib/headers/flags mismatch.
The only thing you should do is ensure the include path is passed to the compiler right before the system ones.
remove the offending framework/path/library (e.g. zlib sometimes packages libpng too!)
include a copy of libpng in your repo and use it
Requirements
the header version matches the version used to build the libpng library
also it is mentioned in bounty description, that solution shouldn't involve PkgConfig
Since PkgConfig is usually the preferred solution, two solutions are presented - one with and one without using PkgConfig.
The first requirement can be represented in some C code like this:
#include <stdio.h>
#include <png.h>
int main(void) {
printf("libpng version of lib (%u):%s",
png_access_version_number(),
png_get_header_version(NULL));
printf("used libpng version in app (%d):%s",
PNG_LIBPNG_VER,
PNG_HEADER_VERSION_STRING);
return 0;
}
Build
To get some debug output, you could use a small script that creates the application:
#!/bin/zsh
rm -rf build
cmake -B build
cmake --build build -v
build/png_app
Solution with PkgConfig
cmake_minimum_required(VERSION 3.17)
project(png_app C)
set(CMAKE_C_STANDARD 99)
find_package(PkgConfig REQUIRED)
pkg_check_modules(PNG libpng16 REQUIRED)
add_executable(png_app main.c)
target_include_directories(png_app PRIVATE ${PNG_INCLUDE_DIRS})
target_link_directories(png_app PRIVATE ${PNG_LIBRARY_DIRS})
target_link_libraries(png_app ${PNG_LIBRARIES})
Test Solution 1
libpng was installed with homebrew like this:
brew install libpng
The output is of the program run is:
libpng version of lib (10637): libpng version 1.6.37 - April 14, 2019
used libpng version in app (10637): libpng version 1.6.37 - April 14, 2019
So we can see that it works like intended! Header version and used library version match.
Note: in the debug output we can see, that cc is called with
.../cc -I/usr/local/Cellar/libpng/1.6.37/include/libpng16 ...
and linking happens like this:
.../cc ... main.c.o -o png_app -L/usr/local/Cellar/libpng/1.6.37/lib -Wl,-rpath,/usr/local/Cellar/libpng/1.6.37/lib -lpng16 -lz
Note: if you get an error during building:
Could NOT find PkgConfig (missing: PKG_CONFIG_EXECUTABLE)
then install it also with brew:
brew install PkgConfig
Solution 2
The second solution is without PkgConfig by using hard coded paths. If a new version of the library is installed, the PNG_BASEPATH must be adapted.
cmake_minimum_required(VERSION 3.17)
project(png_app C)
set(CMAKE_C_STANDARD 99)
set(PNG_BASEPATH /usr/local/Cellar/libpng/1.6.37)
set(PNG_LIBRARIES png16)
add_executable(png_app main.c)
target_include_directories(png_app PRIVATE ${PNG_BASEPATH}/include)
target_link_directories(png_app PRIVATE ${PNG_BASEPATH}/lib)
target_link_libraries(png_app ${PNG_LIBRARIES})
Why does find_package not work?
When I try to use the find_package variant from the OP question: You can see what the problem is with the output of the verbose build command :
The compile command looks like:
.../cc -I/Library/Frameworks/Mono.framework/Headers ... -o .../main.c.o -c .../main.c
So it tries to use libpng14.14 from /Library/Frameworks/Mono.framework instead of the version installed by homebrew.
I met the same issues while setting up the CI on github Actions.
At the end, again, Mono was conflicting. After a few hours trying to work around this conflict using various flags, I still got differences between build and runtime. For those without local MacOs to debug and test, here is a quick solution:
https://gist.github.com/nicerobot/1515915
It removes mono. It does not touch anything else, problem solved. Please do not do that for anything else, this is the most ugliest way to fix things. :)
I am building a package with CMake (3.0.2). This package relies on boost_python and some others. There is an older version in /usr/lib64 and I have another newer version in my own directory /home/x/opt/boost-1.56/lib. There is an symbol link /home/x/opt/boost-1.56/lib64 to deal with the multi-arch gcc.
However, even if I set BOOST_ROOT=/home/x/opt/boost-1.56/lib, and in CMakeCache.txt there is
//Boost python library (release)
Boost_PYTHON_LIBRARY_RELEASE:FILEPATH=/home/x/opt/boost-1.56/lib/libboost_python.a
The generated link.txt contains such a line, instead of the full path:
-Wl,-Bstatic -lboost_python
And when make, it links to the one under /usr/lib64/, which is wrong.
Question:
How to make CMake to use the full path? As documented, only when libraries under system default folders are dealt with -Bstatic, which should not be my case.
Why -Wl,-Bstatic -lboost_python does not work even I have "/home/x/opt/boost-1.56/lib" in LD_LIBRARY_PATH and LIBRARY_PATH? From my limited knowledge they should override the system default.
Information:
gcc --print-search-dir gives:
/home/x/opt/boost-1.56/lib/x86_64-unknown-linux-gnu/4.9.2/
/home/x/opt/boost-1.56/lib/../lib64/
/lib/x86_64-unknown-linux-gnu/4.9.2/
/lib/../lib64/
/usr/lib/x86_64-unknown-linux-gnu/4.9.2/
/usr/lib/../lib64/
/home/x/opt/boost-1.56/lib/
/lib/
/usr/lib/
set(Boost_NO_SYSTEM_PATHS ON)
This will allow it to ignore everything but BOOST_ROOT
Solution (partially only):
CMake uses -Bstatic if there it thinks that the boost libraries it chooses are in the system path, including that related to enviromental variables. Else it use the full path.
I still cannot find why LIBRARY_PATH cannot override the system
So to make it work, REMOVE everything in LIBRARY_PATH and LD_LIBRARY_PATH will do the trick: CMake will use full path in that case.
Add this at the beginn oingf your top level CMakeLists.txt
cmake_minimum_required(VERSION 3.3)
and use cmake 3.3.0 or later.
Interestingly, with cmake 3.11.1 set(Boost_NO_SYSTEM_PATHS ON) did not work for me.
This is kind of an ugly one, but I am stuck so here it goes:
I am writing a dynamic library (henceforth 'mylib.dylib') that uses opencv (highgui and other parts) and is ultimately loaded by ffmpeg (libavfilter). The library compiles just fine. But when I try to dlopen() the dylib from within ffmpeg, I get the following error:
mylib.dylib: dlopen(mylib.dylib, 2): Library not loaded: /usr/local/lib/libpng15.15.dylib
Referenced from: /usr/local/lib/libopencv_highgui.2.3.dylib
Reason: Incompatible library version: libopencv_highgui.2.3.dylib requires version 17.0.0 or later, but libpng15.15.dylib provides version 16.0.0
However, when I run otool -L /usr/local/lib/libpng15.15.dylib I get the following:
/usr/local/lib/libpng15.15.dylib:
/usr/local/lib/libpng15.15.dylib (compatibility version 20.0.0, current version 20.0.0)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.5)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 159.0.0)
Last time I checked, 20.0.0 is later than 17.0.0, which is what highgui said it requires.
I thought maybe it was an architecture problem (I am on a Mac running Lion, so I've encountered a lot of problems with i386/x86_64), but running 'file' on all of the libraries revealed that everything (highgui, libpng, mylib) is Mach-O 64-bit dynamically linked shared library x86_64
I thought perhaps there was some other version of libpng hiding on my system somewhere, but the dlopen error specifically points to /usr/local/lib/libpng15.15.dylib
There is obviously something that I am missing here -- admittedly I am no expert on how these dynamic libraries link. So if I forgot to include some pertinent info, please excuse me.
UPDATE
I forgot to mention -- the lib works fine when I run ffmpeg through XCode (no dlopen error) which leads me to believe that it might have something to do with a environment variable that XCode is setting that I am not.
Thanks!
SOLUTION
1. Don't be dumb
I was using MAMP and since I was calling ffmpeg through a php script, it was using its own set of dylibs, which (surprise surprise) included libpng 16.0.0
I had something similar recently after I uninstalled all my MacPorts and installed updated versions. The problem was the version of libpng (1.2 vs 1.4 vs 1.5), not just the compatibility version. I had built OpenCV against libpng 1.4 (which is libpng14). I thought I could just use install_name_tool to make it look at libpng 1.5 (libpng15) instead but that didn't work. I ended up rebuilding OpenCV against the current versions of the library. That worked fine (after I remembered sudo cmake install).
(Some version numbers above might be incorrect but the gist is accurate.)
So either you don't have the version of libpng you originally had when you built OpenCV or it built against a libpng other than the one in /usr/local/lib (like maybe the one in /opt/local/lib). If that's the case you'll need to make sure you're building against the version you want to use. How to do that depends on your build method (MacPorts or cmake). I use cmake for OpenCV and MacPorts for other libraries which is why the two got out of sync on my system.
Just for those who need the code:
cd /Applications/MAMP/Library/lib
mv libpng15.15.dylib libpng15.15.dylib_old
ln -s /usr/X11/lib/libpng15.15.dylib .
I'm writing a simple C program that uses the TagLib library. I had originally installed the library in /usr/local and compiled and linked my program against it. I've since removed the library completely and am attempting to link against a compiled version of the library in another location. The problem is when I compile my program now, it compiles cleanly, but when attempting to run it, my program is looking for the library that used to exist in /usr/local/lib instead of the new location. For example, my code and the new TagLib library is in /Users/mdi/Code/tag.
I'm compiling my program like this:
cc main.c -Wall -I./taglib/bindings/c -o tag -L./taglib/bindings/c/.libs -ltag_c
Like I said, the compile completes with no errors or warnings. But when attempting to run the binary, I get this error:
dyld: Library not loaded: /usr/local/lib/libtag_c.0.dylib
Referenced from: /Users/mdi/Code/tag/./tag
Reason: image not found
Trace/BPT trap
Running 'otool -L' on the binary shows this:
tag:
/usr/local/lib/libtag_c.0.dylib (compatibility version 1.0.0, current version 1.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 125.2.1)
Why is it still trying to use the TagLib library that no longer even exists in /usr/local/lib and more importantly, how do I fix this?
I don't develop on Mac, but I believe your problem is that your library isn't located in your default dynamic link path. It varies on platform, but /usr/local/lib is a common one amongst them, and that's why it's showing up as the path your linking with at runtime.
You can change your default search path for shard libraries, but I'm uncertain of how to do this on a Mac. You should also be able to set the environment variable DYLD_LIBRARY_PATH within your session to force dyld to search other directories before searching the default directories.
%> export DYLD_LIBRARY_PATH=/path/to/your/library:$DYLD_LIBRARY_PATH
Adding directories to your default search path at a system level is better than the environment variable solution above, but I'm unaware of how to do this on the Mac.
Edit:
After a little searching, I found this link that you'll likely find helpful.
First, are you sure there are no dangling symlinks in /usr/local/bin ? Then, I don't know how that library is compiled/linked so I can't tell you how to do this, but you can try adding the -rpath linker option. For example:
gcc -o libtag_c.0.dylib ... -Wl,-rpath,/Users/mdi/Code/tag
That option adds an explicit search path for the library and the runtime linker will then search at that path for the library.
Thanks for the help guys. Actually, unfortunately neither of these solutions worked. After more digging around, I noticed that the newly built libs were getting the old path as well, as reported by 'otool -L'. I managed to fix this by using install_name_tool -id /new/path/to/lib newly_built_lib on the libs after they were already built.
This seems to fix the problem, but I'm still curious as to why this old path to the lib is still hanging around even after it has been completely removed (no dangling symlinks). I also tried sudo update_dyld_shared_cache -force without success. But thanks for the help anyway.