Statically linking libclang in C code - c

I'm trying to write a simple syntax checker for C code using the frontend available in libclang. Due to deployment concerns, I need to be able to statically link all the libraries in libclang, and not pass around the .so file that has all the libraries.
I'm building clang/llvm from source, and in llvm/Release+Asserts/lib I have a bunch of .a files that I think I should be able to use, but it never seems to work (the linker spews out thousands of errors about missing symbols). However, when I compile it using the libclang.so also present in that directory as follows:
clang main.c -o bin/dlc -I../llvm/tools/clang/include -L../llvm/Release+Asserts/lib/ -lclang
Everything seems to work well.
What is the minimum set of .a files I need to include to make this work? I've tried including absolutely all of the .a files in the build output directory, with them provided to clang/gcc in different orders, without any success. I only need the functions mentioned in libclang's Index.h, but there don't seem to be any resources or documentation on what the various libclang*.a files are for. It would be very helpful to know which files libclang.so pulls in.

The following is supposed to work, as long the whole project has all static libraries (I counted 116 in my Release/lib directory).
clang main.c -o bin/dlc -I../llvm/tools/clang/include ../llvm/Release/lib/*.a
[edit: clang main.c -o bin/dlc -I../llvm/tools/clang/include ../llvm/Release/lib/libclang.a ../llvm/Release/lib/*.a]
Note that the output binary is not static, so you don't need any -static flag for gcc or ld, if you're using this syntax.
If that doesn't work you might need to list the libraries in order: if some library requires a function available in another library, then it may be necessary to list it first in the command line. See comments about link order at:
http://gcc.gnu.org/onlinedocs/gcc-4.7.2/gcc/Link-Options.html#Link-Options

Related

C - error: conflicting types for 'SDL_main' [duplicate]

I want to make a game using SDL2, but I'm unable to compile and/or run my code, please help!
SDL2 is notoriously hard to set up, and it's often the first library aspiring game developers try to use.
This post is intended as a canonical duplicate for common problems with setting up SDL2.
This answer is about MinGW / GCC, and not Visual Studio.
This answer only applies to Windows.
Common errors
The common errors are:
SDL.h: No such file or directory (when compiling)
Various SDL_main problems: "undefined reference to SDL_main", "conflicting types for SDL_main" or "number of arguments doesn't match prototype", etc. (when compiling or linking)
undefined reference to other functions (when linking)
DLL problems: (when running your program)
'??.dll' was not found
procedure entry point ... could not be located in ..., and other mysterious DLL-related errors
The program seemingly doing nothing when launched
This list is sorted from bad to good. If you change something and get a different error, use this list to tell if you made things better or worse.
The preamble
0. Don't follow bad advice.
Some resources will suggest you to do #define SDL_MAIN_HANDLED or #undef main. Don't blindly follow that advice, it's not how SDL2 is intended to be used.
If you do everything correcty, it will never be necessary. Learn the intended approach first. Then you can research what exactly that does, and make an educated decision.
1. Figure out how to compile directly from the console, you can start using an IDE and/or build system later.
If you're using an IDE, I suggest to first make sure you're able to compile your program directly from the console, to rule out any IDE configuration problems. After you figure that out, you can use the same compiler options in your IDE.
The same applies to build systems, such as CMake.
2. Download the right SDL2 files. Make sure you have the right files. You need the archive called SDL2-devel-2.0.x-mingw.tar.gz from here.
Extract it to any directory, preferably somewhere near your source code. Extracting into the compiler installation directory is often considered a bad practice (and so is copying them to C:\Windows, which is a horrible idea).
3. Know the difference between compiler flags and linker flags. A "flag" is an option you specify in the command line when building your program. When you use a single command, e.g. g++ foo.cpp -o foo.exe, all your flags are added to the same place (to this single command).
But when you build your program in two steps, e.g.:
g++ foo.cpp -c -o foo.o (compiling)
g++ foo.o -o foo.exe (linking)
you have to know which of the two commands to add a flag to. Those are "compiler flags" and "linker flags" respectively.
Most IDEs will require you to specify compiler and linker flags separately, so even if you use a single command now, it's good to know which flag goes where.
Unless specified otherwise, the order of the flags doesn't matter.
SDL.h: No such file or directory
Or any similar error related to including SDL.h or SDL2/SDL.h.
You need to tell your compiler where to look for SDL.h. It's in the SDL files you've downloaded (see preamble).
Add -Ipath to your compiler flags, where path is the directory where SDL.h is located.
Example: -IC:/Users/HolyBlackCat/Downloads/SDL2-2.0.12/x86_64-w64-mingw32/include/SDL2. Relative paths work too, e.g. -ISDL2-2.0.12/x86_64-w64-mingw32/include/SDL2.
Note that the path will be different depending on how you write the #include:
If you do #include <SDL.h>, then the path should end with .../include/SDL2 (like above). This is the recommended way.
If you do #include <SDL2/SDL.h>, then the path should end with .../include.
Various SDL_main problems
You can get several different errors mentioning SDL_main, such as undefined reference to SDL_main, or conflicting types for 'SDL_main', or number of arguments doesn't match prototype, etc.
You need to have a main function. Your main function must look like int main(int, char **). NOT int main() and NOT void main(). This is a quirk of SDL2, related to it doing #define main SDL_main.
Adding parameter names is allowed (and is mandatory in C), e.g. int main(int argc, char **argv). Also the second parameter can be written as char *[] or with a name: char *argv[]. No other changes are allowed.
If your project has multiple source files, make sure to include SDL.h in the file that defines the main function, even if it doesn't otherwise use SDL directly.
Try to avoid #define SDL_MAIN_HANDLED or #undef main when solving this issue, see preamble for explanation.
undefined reference to various functions
• undefined reference to SDL_...
The error message will mention various SDL_... functions, and/or WinMain. If it mentions SDL_main, consult the section "Various SDL_main problems" above. If the function names don't start with SDL_, consult the section "undefined reference to other functions" below.
You need to add following linker flags: -lmingw32 -lSDL2main -lSDL2 -Lpath, where path is the directory where libSDL2.dll.a and libSDL2main.a (which you've downloaded) are located. The order of the -l... flags matters. They must appear AFTER any .c/.cpp/.o files.
Example: -LC:/Users/HolyBlackCat/Desktop/SDL2-2.0.12/x86_64-w64-mingw32/lib. Relative paths work too, e.g. -LSDL2-2.0.12/x86_64-w64-mingw32/lib.
When you use -l???, the linker will look for a file called lib???.dll.a or lib???.a (and some other variants), which is why we need to pass the location of those files. libmingw32.a (corresponding to -lmingw32) is shipped with your compiler, so it already knows where to find it.
I added all those flags and nothing changed, or I'm getting skipping incompatible X when searching for Y:
You probably use the wrong SDL .a files. The archive you downloaded contains two sets of files: i686-w64-mingw32 (32-bit) and x86_64-w64-mingw32 (64-bit). You must use the files matching your compiler, which can also be either 32-bit or 64-bit.
Print (8*sizeof(void*)) to see if your compiler is 32-bit or 64-bit.
Even if you think you use the right files, try the other ones to be sure.
Some MinGW versions can be switched between 32-bit and 64-bit modes using -m32 and -m64 flags (add them to both compiler and linker flags).
I get undefined reference to a specific function:
• undefined reference to WinMain only
There are several possibilities, all of which were covered in the previous section:
You forgot -lmingw32 and/or -lSDL2main linker flags.
You must use following linker flags, in this exact order, after
any .c/.cpp/.o files: -lmingw32 -lSDL2main -lSDL2
The libSDL2main.a file you use doesn't match your compiler (32-bit file with a 64-bit compiler, or vice versa).
Try to avoid #define SDL_MAIN_HANDLED or #undef main when solving this issue, see preamble for explanation.
• undefined reference to SDL_main only
See the section "Various SDL_main problems" above.
• undefined reference to other functions
Your linker found and used libSDL2.a, but it should be finding and using libSDL2.dll.a. When both are available, it prefers the latter by default, meaning you didn't copy the latter to the directory you passed to -L.
If you intended to perform static linking, see the section called "How do I distribute my app to others?" below.
Nothing happens when I try run my app
Let's say you try to run your app, and nothing happens. Even if you try to print something at the beginning of main(), it's not printed.
Windows has a nasty habit of not showing some DLL-related errors when the program is started from the console.
If you were running your app from the console (or from an IDE), instead try double-clicking the EXE in the explorer. Most probably you'll now see some DLL-related error; then consult one of the next sections.
??.dll was not found
Copy the .dll mentioned in the error message, and place it next to your .exe.
If the DLL is called SDL2.dll, then it's in the SDL files you've downloaded (see preamble). Be aware that there are two different SDL2.dlls: a 32-bit one (in the i686-w64-mingw32 directory), and a 64-bit one (in x86_64-w64-mingw32). Get the right one, if necessary try both.
Any other DLLs will be in your compiler's bin directory (the directory where gcc.exe is located).
You might need to repeat this process 3-4 times, this is normal.
For an automatic way of determining the needed DLLs, see the next section.
procedure entry point ... could not be located in ... and other cryptic DLL errors
Your program needs several .dlls to run, and it found a wrong version of one, left over from some other program you have installed.
It looks for DLLs in several different places, but the directory with the .exe has the most priority.
You should copy all DLLs your program uses (except the system ones) into the directory where your .exe is located.
A reliable way to get a list of needed DLLs is to blindly copy a bunch of DLLs, and then remove the ones that turn out to be unnecessary:
Copy SDL2.dll. It's in the SDL files you've downloaded (see preamble). Be aware that there are two different SDL2.dlls: a 32-bit one (in the i686-w64-mingw32 directory), and a 64-bit one (in x86_64-w64-mingw32). Get the right one, if necessary try both.
Copy all DLLs from your compiler's bin directory (the directory where gcc.exe is located).
Now your program should run, but we're not done yet.
Download NTLDD (or some other program that displays a list of used DLLs). Run ntldd -R your_program.exe.
Any DLL not mentioned in its output should be removed from the current directory. Your program uses everything that remains.
I ended up with following DLLs, expect something similar: SDL2.dll, libgcc_s_seh-1.dll, libstdc++-6.dll (C++ only), libwinpthread-1.dll.
Can I determine the needed DLLs without copying excessive ones?
Yes, but it's less reliable.
Your program searches for DLLs in following locations, in this order:
The directory where your .exe is located.
C:\Windows, including some of its subdirectories.
The directories listed in PATH.
Assuming you (or some jank installer) didn't put any custom DLLs into C:\Windows, adding your compiler's bin directory to the PATH (preferably as the first entry) and either putting SDL2.dll in the same directory as the .exe or into some directory in the PATH should be enough for your program to work.
If this works, you can then run ntldd without copying any DLLs beforehand, and copy only the necessary ones. The reason why you'd want to copy them at all at this point (since your app already works) is to be able to distribute it to others, without them having to install the compiler for its DLLs. Skip any DLLs located outside of your compiler's bin directory (except for SDL2.dll).
Note that the possibility of having weird DLLs in C:\Windows is real. E.g. Wine tends to put OpenAL32.dll into C:\Windows, so if you try this process with OpenAL on Wine, it will fail. If you're making a sciprt that runs ntldd automatically, prefer copying the DLLs (or at least symlinking them - I heard MSYS2 can emulate symlinks on Windows?).
Can I make an EXE that doesn't depend on any DLLs?
It's possible to make an .exe that doesn't depend on any (non-system) .dlls by using the -static linker flag, this is called "static linking". This is rarely done, and you shouldn't need to do this if you did the above steps correctly. This requires some additional linker flags; they are listed in file ??-w64-mingw32/lib/pkgconfig/sdl2.pc shipped with SDL, in the Libs.private section. Notice that there are two files, for x32 and x64 respectively.
How do I distribute my app to others?
Follow the steps in the previous section, titled procedure entry point ... could not be located in ....
A saner alternative?
There is MSYS2.
It has a package manager that lets you download prebuilt libraries, and, as a bonus, a fresh version of the compiler.
Install SDL2 from its package manager. Use a tool called pkg-config (also from the package manager) to automatically determine all necessary flags (pkg-config --cflags SDL2 for compiler flags, pkg-config --libs SDL2 for linker flags).
This is the same experience as you would have on Linux (maybe except for some DLL management hassle).
Bonus - Other problems
Q: My program always opens a console window when I run it, how do I hide it?
A: Add -mwindows to the linker flags.
Q: I get error 'SDL_VideoMode' wasn't declared in this scope.
A: SDL_VideoMode is from SDL1.2, it's not a part of the newer SDL2. Your code was written for the outdated version of SDL. Find a better tutorial that deals specifically with SDL2.
Q: My program has the default file icon, but I want a custom one.
A: Your icon must be in the .ico format. If your graphics editor doesn't support it, make a series of .pngs of common sizes (e.g. 16x16, 32x32, 48x48, 64x64), then convert them to a single .ico using ImageMagick: magick *.png result.ico (or with convert instead of magick).
Create a file with the .rc extension (say, icon.rc), with following contents MyIconName ICON "icon.ico" (where MyIconName is an arbitrary name, and "icon.ico" is the path to the icon). Convert the file to an .o using windres -O res -i icon.rc -o icon.o (the windres program is shipped with your compiler). Specify the resulting .o file when linking, e.g. g++ foo.cpp icon.o -o foo.exe.
Recent versions of SDL2 have a nice property of using the same icon as the window icon, so you don't have to use SDL_SetWindowIcon.
A solution for Visual Studio:
Why not use a package manager? I use vcpkg, and it makes super easy to consume 3rd party libraries. Grab the vcpkg source, and extract it to a safe place, like C:/, then run its bootstrap script bootstrap-vcpkg.bat, this will generate vcpkg executable. Then run vcpkg integrate install to make libraries installed with vcpkg available in Visual Studio.
Search for the library you need:
vcpkg search sdl
imgui[sdl2-binding] Make available SDL2 binding
libwebp[vwebp-sdl] Build the vwebp viewer tool.
magnum[sdl2application] Sdl2Application library
sdl1 1.2.15#12 Simple DirectMedia Layer is a cross-platform development library designed to p...
sdl1-net 1.2.8-3 Networking library for SDL
sdl2 2.0.12-1 Simple DirectMedia Layer is a cross-platform
...
Install it with: vcpkg install sdl2.
Now you just need include SDL2 headers, and everything will work out of the box. The library will be linked automatically.
You can learn more about vcpkg here.
On Mac this is what I follow for XCode (must install g++):
sdl linking:
g++ main.cpp -o main $(sdl2-config --cflags --libs)
XCODE project steps:
open terminal app (macOS)
BUILD SETTINGS (select 'all' and 'combined' search bar enter: "search")
click on "header search paths(way right side click)
add: /usr/local/include
BUILD PHASES --> LINK BINARY LIBRARIES (click plus)
type in SDL --> click "add other"
press: command+SHIFT+g (to bring search bar)
type in: usr/local/Cellar
navigate to: SDL2 -->2.0.8 -->lib --> libSDL2-2.2.0.dylib (make sure not shortcut)

Linking shared libraries with gcc on Linux

I need to compile and, most importantly, link a C program that uses a proprietary function present in a shared library file. Because of lack of communication with the previous development team, there is no proper documentation. I declared a function prototype (because I know the number and type of arguments):
int CustomFunction(unsigned char *in, int size);
Since that function name can be grepped from /customlibs/libcustom.so, I tried to compile the code and link it like this:
gcc -L/customlibs testing.c -o testing -lcustom
Which throws a few error messages looking like this:
/customlibs/libcustom.so: undefined reference to `AnotherCustomFunction'
Obviously, I need to tell linker to include other libraries as well, and, to make things worse, they need to be in certain order. I tried exporting LD_LIBRARY_PATH, using -Wl,-rpath=, -Wl,--no-undefined and -Wl,--start-group. Is there an easy way to give the linker all the .so files without the proper order?
I found the solution (or a workaround) to my problem: adding -Wl,--warn-unresolved-symbols, which turns errors to warnings. Note that this works only if you are ABSOLUTELY certain your function does not depend on the symbols mentioned in undefined refernce to: messages.
Add them on the command line is a way to do it. Something like this below. The LD_LIBRARY_PATH tells gcc where to look for libraries, but you still need to say what libraries to include.
gcc -L/customlibs testing.c -o testing -lcustom -lmylib1 -lmylib2 -lmylib3
You should also include all the header files of your shared library by adding the -I option of gcc, for example : gcc [...] -I/path/to/your/lib/header/files [...]

GCC - Adding Libraries

I want to use functions in the header files gmp.h and mpfr.h, which are in the file /opt/local/include.
But when I run gcc with -v, all of the search paths are something like /Application/Xcode.app/Contents/etc.
I have tried adding LD_LIBRARY_PATH="/opt/local/include" to .bash_profile but it doesn't work. The compiler either tells me that 'gmp.h' file not found, or Undefined symbols for architecture x86_64.
What should I do?
Converting comments into an answer.
You need to add -I/opt/local/include to compile commands (to specify where the headers are) and -L/opt/local/lib and -lgmp and -lmpfr (possibly in the reverse order — MPFR before GMP) to link commands.
That works! Would you mind explaining a little bit the logic behind this? For example if I had another header file header.h I need, how should I include it?
You include it with #include "header.h". You compile the code with -I/directory/containing/header to find the header. You specify where the library (libheader.a or libheader.dylib, since you seem to be on macOS) is too, with -L/directory/containing/lib and -lheader — or whatever is appropriate.
The -I tells the preprocessor to look in the named directory for header files, so it looks for /directory/containing/header/header.h, for example.
The -L tells the linker where to find libraries (so it looks for /directory/containing/lib/libheader.dylib etc).
The -lheader tells the linker to look for libheader.a or libheader.dylib (or local equivalents) for the libraries.
Except for the use of .dylib vs .so vs .dll vs … (and .a vs .lib vs …), the same principles apply to other systems too.
This is probably a duplicate.

Is using --start-group and --end-group when linking faster than creating a static library?

If one builds static libraries in one's build scripts and one wants to use those static libraries in linking the final executable, the order one mentions the .a files is important:
g++ main.o hw.a gui.a -o executable
If gui.a uses something defined in hw.a the link will fail, because at the time hw.a is processed, the linker doesn't yet know that the definition is needed later, and doesn't include it in the being.generated executable. Manually fiddling around with the linker line is not practical, so a solution is to use --start-group and --end-group which makes the linker run twice through the libraries until no undefined symbols are found anymore.
g++ main.o -Wl,--start-group hw.a gui.a -Wl,--end-group -o executable
However the GNU ld manual says
Using this option has a significant performance cost. It is best to use it only when there are unavoidable circular references between two or more archives.
So I thought that it may be better to take all .a files and put them together into one .a file with an index (-s option of GNU ar) which says in what order the files need to be linked. Then one gives only that one .a file to g++.
But I wonder whether that's faster or slower than using the group commands. And are there any problems with that approach? I also wonder, is there better way to solve these interdependency problems?
EDIT: I've written a program that takes a list of .a files and generates a merged .a file. Works with the GNU common ar format. Packing together all static libs of LLVM works like this
$ ./arcat -o combined.a ~/usr/llvm/lib/libLLVM*.a
I compared the speed against unpacking all .a files manually and then putting them into a new .a file using ar, recomputing the index. Using my arcat tool, I get consistent runtimes around 500ms. Using the manual way, time varies greatly, and takes around 2s. So I think it's worth it.
Code is here. I put it into the public domain :)
You can determine the order using the lorder and tsort utilities, for example
libs='/usr/lib/libncurses.a /usr/lib/libedit.a'
libs_ordered=$(lorder $libs | tsort)
resulting in /usr/lib/libedit.a /usr/lib/libncurses.a because libedit depends on libncurses.
This is probably only a benefit above --start-group if you do not run lorder and tsort again for each link command. Also, it does not allow mutual/cyclic dependencies like --start-group does.
Is there a third option where you just build a single library to begin with? I had a similar problem and I eventually decided to go with the third option.
In my experience, group is slower than just unifying the .a files. You can extract all files from the archive, then create a new .a file from from the smaller files
However, you have to be careful about a circumstance where both files happen to contain the same definition (you can explicitly check for this by using nm to see what definitions are contained in each library)

link c function in nasm

got a nasm project and i'm calling a c function from it
I put the name of the function in "extern"
and when linking i put all the links together but i can an error of "undefined reference to"
here is my compile/link command
gcc -o Project4 Project4.o array1c.c readdouble.o writedouble.o readarray.o printarray.o addarray.o invertarray.o invertarray2.o invertarray3.o averagearray.o quicksort.c
I would first compile all of your .c files using the "gcc -c" command into object files, then link those resulting .o files (such as "array1c.o" and "quicksort.o") together with your other pre-existing object files and see if that still gives you an undefined reference. That may be an unnecessary step, but I've never combined raw .c files and .o files in a single call to gcc.
You may also have to add an underscore to the beginning of any c-functions called ... I know this an be a platform dependent thing (i.e., Linux typically doesn't need underscores on c-functions whereas OSX and some other UNIX platforms do).
Lastly you could try, using ld, to just link all the object files together at once rather than linking some of the object files together into Project4.o, and then linking that to what you had assembled using nasm (at least that's what I'm assuming you're doing, i.e., you're making a Project4.o, and then calling functions from that in your assembly code).
Hope this helps,
Jason

Resources