Today, while working with one custom library, I found a strange behavior.
A static library code contained a debug main() function. It wasn't inside a #define flag. So it is present in library also. And it is used link to another program which contained the real main().
When both of them are linked together, the linker didn't throw a multiple declaration error for main(). I was wondering how this could happen.
To make it simple, I have created a sample program which simulated the same behavior:
$ cat prog.c
#include <stdio.h>
int main()
{
printf("Main in prog.c\n");
}
$ cat static.c
#include <stdio.h>
int main()
{
printf("Main in static.c\n");
}
$ gcc -c static.c
$ ar rcs libstatic.a static.o
$ gcc prog.c -L. -lstatic -o 2main
$ gcc -L. -lstatic -o 1main
$ ./2main
Main in prog.c
$ ./1main
Main in static.c
How does the "2main" binary find which main to execute?
But compiling both of them together gives a multiple declaration error:
$ gcc prog.c static.o
static.o: In function `main':
static.c:(.text+0x0): multiple definition of `main'
/tmp/ccrFqgkh.o:prog.c:(.text+0x0): first defined here
collect2: ld returned 1 exit status
Can anyone please explain this behavior?
Quoting ld(1):
The linker will search an archive only once, at the location where it is specified on the command line. If the archive defines a symbol which was undefined in some object which appeared before the archive on the command line, the linker will include the appropriate file(s) from the archive.
When linking 2main, main symbol gets resolved before ld reaches -lstatic, because ld picks it up from prog.o.
When linking 1main, you do have undefined main by the time it gets to -lstatic, so it searches the archive for main.
This logic only applies to archives (static libraries), not regular objects.
When you link prog.o and static.o, all symbols from both objects are included unconditionally, so you get a duplicate definition error.
When you link a static library (.a), the linker only searches the archive if there were any undefined symbols tracked so far. Otherwise, it doesn't look at the archive at all. So your 2main case, it never looks at the archive as it doesn't have any undefined symbols for making the translation unit.
If you include a simple function in static.c:
#include <stdio.h>
void fun()
{
printf("This is fun\n");
}
int main()
{
printf("Main in static.c\n");
}
and call it from prog.c, then linker will be forced to look at the archive to find the symbol fun and you'll get the same multiple main definition error as linker would find the duplicate symbol main now.
When you directly compile the object files(as in gcc a.o b.o), the linker doesn't have any role here and all the symbols are included to make a single binary and obviously duplicate symbols are there.
The bottom line is that linker looks at the archive only if there are missing symbols. Otherwise, it's as good as not linking with any libraries.
After the linker loads any object files, it searches libraries for undefined symbols. If there are none, then no libraries need to be read. Since main has been defined, even if it finds a main in every library, there is no reason to load a second one.
Linkers have dramatically different behaviors, however. For example, if your library included an object file with both main () and foo () in it, and foo was undefined, you would very likely get an error for a multiply defined symbol main ().
Modern (tautological) linkers will omit global symbols from objects that are unreachable - e.g. AIX. Old style linkers like those found on Solaris, and Linux systems still behave like the unix linkers from the 1970s, loading all of the symbols from an object module, reachable or not. This can be a source of horrible bloat as well as excessive link times.
Also characteristic of *nix linkers is that they effectively search a library only once for each time it is listed. This puts a demand on the programmer to order the libraries on the command line to a linker or in a make file, in addition to writing a program. Not requiring an ordered listing of libraries is not modern. Older operating systems often had linkers that would search all libraries repeatedly until a pass failed to resolve a symbol.
Related
I have following two source codes and want to link them.
// test.c
#include <stdio.h>
void lib2();
void lib1(){
lib2();
return 0;
}
// main.c
#include <stdio.h>
int main() {
return 0;
}
I've used gcc -c main.c and gcc -c test.c to generate objects files
$ ls *.o
main.o test.o
and I've used ar rcs test.a test.o command to generate static library(test.a) from object file test.o
Then, I tried to build executable by linking main.o with test.a or test.o. As far as I know, a static library file(.a extension) is a kind of simple collection of object files(.o). so I expected both would give same result: error or success. but it didn't.
Linking with the object file gives undefined reference error.
$ gcc -o main main.o test.o
/usr/bin/ld: test.o: in function `lib1':
test.c:(.text+0xe): undefined reference to `lib2'
collect2: error: ld returned 1 exit status
$
but linking with the static library doesn't give any error and success on compilation.
$ gcc -o main main.o test.a
$
Why is this happening? and how can I get undefined reference errors even when linking with static libraries?
If your code contains a function call expression then the language standard requires a function definition exists. (See C11 6.9/3). If you don't provide a definition then it is undefined behaviour with no diagnostic required .
The rule was written this way so that implementation vendors aren't forced to perform analysis to determine if a function is ever called or not; for example in your library scenario the compiler isn't forced to dig around in the library if none of the rest of the code contains anything that references that library.
It's totally up to the implementation what to do, and in your case it decides to give an error in one case and not the other. To avoid this, you can provide definitions for all the functions you call.
You might be able to modify the behaviour in the first case by using linker options such as elimination of unused code sections. Another thing you can do is call lib1() from main() -- this is still not guaranteed to produce an error but is more likely to.
Force the linker to do some work use -flto option and the error will go away.
ld does not search libraries for objects which are not used it only searches for symbols used in object files. Imagine that you have a library where some functions require defined callbacks. If you do not have them in every program you link against the library even if you do not use those functions.
I expected both would give same result: error or success. but it didn't.
Your expectation is incorrect. A good explanation of the difference between .o and .a with respect to linking is here.
I'm getting the following error and can't for the life of me figure out what I'm doing wrong.
$ gcc main.c -o main
Undefined symbols:
"_wtf", referenced from:
_main in ccu2Qr2V.o
ld: symbol(s) not found
collect2: ld returned 1 exit status
main.c:
#include <stdio.h>
#include "wtf.h"
main(){
wtf();
}
wtf.h:
void wtf();
wtf.c:
void wtf(){
printf("I never see the light of day.");
}
Now, if I include the entire function in the header file instead of just the signature, it complies fine so I know wtf.h is being included. Why doesn't the compiler see wtf.c? Or am I missing something?
Regards.
You need to link wtf with your main. Easiest way to compile it together - gcc will link 'em for you, like this:
gcc main.c wtf.c -o main
Longer way (separate compilation of wtf):
gcc -c wtf.c
gcc main.c wtf.o -o main
Even longer (separate compilation and linking)
gcc -c wtf.c
gcc -c main.c
gcc main.o wtf.o -o main
Instead of last gcc call you can run ld directly with the same effect.
You are missing the fact that merely including a header doesn't tell the compiler anything about where the actual implementation (the definitions) of the things declared in the header are.
They could be in a C file next to the one doing the include, they could come from a pre-compiled static link library, or a dynamic library loaded by the system linker when reading your executable, or they could come at run-time user programmer-controlled explicit dynamic loading (the dlopen() family of function in Linux, for instance).
C is not like Java, there is no implicit rule that just because a C file includes a certain header, the compiler should also do something to "magically" find the implementation of the things declared in the header. You need to tell it.
I am using Cygwin. I have two files in the same directory, test.c and iah202_graphics.h. test.c uses functions from the header file, where I have used #include "iah202_graphics.h". I have added the Cygwin directory to my Environment Variables (PATH) already.
However I receive these errors for every function call:
$ gcc -o test test.c
/cygdrive/c/Users/Matthew/AppData/Local/Temp/cclm2bNk.o:test.c:(.text+0x27): undefined reference to `draw_line'.
/cygdrive/c/Users/Matthew/AppData/Local/Temp/cclm2bNk.o:test.c:(.text+0x27): relocation truncated to fit:
R_X86_64_PC32 against undefined symbol `draw_line'.
/cygdrive/c/Users/Matthew/AppData/Local/Temp/cclm2bNk.o:test.c:(.text+0x4a): undefined reference to
`draw_line'.
collect2: error: ld returned 1 exit status.
It's having trouble linking to the header file even though I've simply stated which file to use in the local directory. I don't understand what I'm doing wrong?
Undefined reference to 'blah' is a linker error rather than a compiler error and is almost always caused by not including a needed library.
Including a header file in your source file does not usually link in the code required to provided the functions declared in that header.
For example, were you to prevent linking of the C runtime library, you could include stdio.h as many times as you wanted to, and still not be able to resolve printf.
Bottom line, you generally need two steps:
include the relevant header file in your source code so it knows about the declarations of things provided; and
link against the relevant library or object file so it has access to the definitions of the things provided.
That could be something as simple as:
gcc -o test -I/path/to/iah202includes test.c -L/path/to/iah202libs -liah202
where -I indicates where include files can be found, -L adjusts the search path for library files, and -l actually specifies the library file to use.
Even simpler is if you have the source file for the graphics stuff (which seems to be the case based on your comments). In that case no library is needed, you can simply use:
gcc -o test test.c iab202_graphics.c
and that will compile both those translation units then link them together.
I have two directories, sorting and searching (children of the same directory), that have .c source files and .h header files:
mbp:c $ ls sorting
array_tools.c bubble_sort.c insertion_sort.c main selection_sort.c
array_tools.h bubble_sort.h insertion_sort.h main.c selection_sort.h
mbp:c $ ls searching
array_tools.c array_tools.h binary_search.c binary_search.h linear_search.c linear_search.h main main.c
Within searching, I am building an executable that needs to use insertion_sort function, declared in insertion_sort.h and defined in insertion_sort.c inside sorting. The following compilation successfully produces an executable:
mbp:searching $ clang -Wall -pedantic -g -iquote"../sorting" -o main main.c array_tools.c binary_search.c linear_search.c ../sorting/insertion_sort.c
However, I would like to be able to include functions from arbitrary directories by including a header using #include and then providing the compiler with the search path. Do I need to precompile the .c files to .o files beforehand? The man page for clang lists the following option:
-I<directory>
Add the specified directory to the search path for include files.
But the following compilation fails:
mbp:searching $ clang -Wall -pedantic -g -I../sorting -o main main.c array_tools.c binary_search.c linear_search.c
Undefined symbols for architecture x86_64:
"_insertion_sort", referenced from:
_main in main-1a1af0.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
main.c has the following includes:
#include <stdio.h>
#include <stdlib.h>
#include "linear_search.h"
#include "binary_search.h"
#include "array_tools.h"
#include "insertion_sort.h"
I do not understand the link between header files, source files, and object files. To include a function defined in a .c file, is it sufficient to include the homonymous header file, given that the .c file is in the same directory as the header? I have read multiple answers here on SO, the man page for clang and a number of tutorials, but was unable to find a definitive, clear answer.
In response to #spectras:
One by one, you give the compiler a source file to work on. For instance:
cc -Wall -Ipath/to/some/headers foo.c -o foo.o
Running
mbp:sorting $ clang -Wall insertion_sort.c -o insertion_sort.o
produces the following error:
Undefined symbols for architecture x86_64:
"_main", referenced from:
implicit entry/start for main executable
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Okay, it's mixed up a bit. Let's see how one typically compiles a simple multi-file project.
One by one, you give the compiler a source file to work on. For instance:
cc -c -Wall -Ipath/to/some/headers foo.c -o foo.o
The -c flag tells the compiler you want an object file, so it should not run the linker.
The compiler runs the preprocessor on the source file. Among other things, every time it sees a #include directive, it searches the include paths for named file and basically copy-pastes it, replacing the #include with the content. This is done recursively.
This is the step where all .h you include get merged into the source file. We call the whole thing a translation unit.
You can see the result of this step by using -E flag and inspect the result, for instance:
cc -Wall -Ipath/to/some/headers foo.c -E -o foo.test
Let's make this short as other steps are not relevant to your question. The compiler then creates an object file from the resulting source code. The object file contains binary version of all code and data that was in the translation unit, plus metadata that will be used to put everything together and some other stuff (like debugging info).
You can inspect the contents of an object file using objdump -xd foo.o.
Note that as this is done for each source file, this means that headers get parsed and compiled again and again and again. That's the reason they should only declare stuff and not contain actual code: you would end up with that code in every object file.
Once done, you link all the object files into an executable, for instance:
cc foo.o bar.o baz.o -o myprogram
This step will gather all, resolve dependencies and write everything into an executable binary. You may also pull in external object files using -l, like when you do -lrt or -lm.
For instance:
foo.c includes bar.h
bar.h contains a declaration of function do_bar: void do_bar(int);
foo.c can use it, and compiler will generate foo.o correctly
foo.o will have placeholders and the information that it requires do_bar
bar.c defines the implementation of do_bar.
so bar.o will have the information “hey if anyone needs do_bar, I got it here”.
linking step will replace placeholders with actual calls to do_bar.
Finally, when you pass multiple .c files to the compiled like you do in your question, the compiler does basically the same thing, only it won't generate the intermediate object files. Overall process behaves the same though.
So, what about your error?
Undefined symbols for architecture x86_64:
"_insertion_sort", referenced from:
_main in main-1a1af0.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
See? It says linking step failed. That means previous step went well. The #include worked. It's just in the linking step, it's looking for a symbol (data or code) called _insertion_sort, and does not find it. That's because that symbol was declared somewhere (otherwise source using it would not have compiled), but its definition is not available. Either no source file implemented it, or the object file that contains it was not given to the linker.
=> You need to make _insertion_sort's definition available. Either by adding ../sorting/insertion_sort.c to the source lists you pass or by compiling it into an object file and passing that. Or by building it into a library so it can be shared by your two binaries (otherwise they'll each have a copy embedded).
When you get there, usually starting to use a build toolsuite such as CMake is a good idea. It will take care of all the details for you.
I'm getting the following error and can't for the life of me figure out what I'm doing wrong.
$ gcc main.c -o main
Undefined symbols:
"_wtf", referenced from:
_main in ccu2Qr2V.o
ld: symbol(s) not found
collect2: ld returned 1 exit status
main.c:
#include <stdio.h>
#include "wtf.h"
main(){
wtf();
}
wtf.h:
void wtf();
wtf.c:
void wtf(){
printf("I never see the light of day.");
}
Now, if I include the entire function in the header file instead of just the signature, it complies fine so I know wtf.h is being included. Why doesn't the compiler see wtf.c? Or am I missing something?
Regards.
You need to link wtf with your main. Easiest way to compile it together - gcc will link 'em for you, like this:
gcc main.c wtf.c -o main
Longer way (separate compilation of wtf):
gcc -c wtf.c
gcc main.c wtf.o -o main
Even longer (separate compilation and linking)
gcc -c wtf.c
gcc -c main.c
gcc main.o wtf.o -o main
Instead of last gcc call you can run ld directly with the same effect.
You are missing the fact that merely including a header doesn't tell the compiler anything about where the actual implementation (the definitions) of the things declared in the header are.
They could be in a C file next to the one doing the include, they could come from a pre-compiled static link library, or a dynamic library loaded by the system linker when reading your executable, or they could come at run-time user programmer-controlled explicit dynamic loading (the dlopen() family of function in Linux, for instance).
C is not like Java, there is no implicit rule that just because a C file includes a certain header, the compiler should also do something to "magically" find the implementation of the things declared in the header. You need to tell it.