I'm working on an embedded project and I am duplicating a sample project.
Not taking the linking order of the object files into account, I just put the c files in a random order in my Makefile.
Compiling and linking yields an executable.elf of 1.9Mb.
No errors were generated but the executable didn't work.
After a long search with no solution I finally duplicated the project exactly, including the order of the c files (120 of them) and behold I got an executable.elf of 2.2Mb and no errors. AND the executable worked.
Nothing changed in compile-options and/or linking-options. Just changed the order in which the c files are listed in the makefile, and therefore the order of the object files at link time.
I suspect that there are multiple duplicate function implementations with different bodies/sizes. My hypothesis is that link time the linker, with no memory of previous link actions, just picks the first one it encounters without raising an error.
However I would like to get a hold on this provided library (all single c files, no lib *.a file) and find the duplicate function implementations. So I know in which order I should provide the c files and more importantly, why.
Two questions:
Is the description above, indeed a potential cause of the issues?
Are there other possibilities?
How to find the duplicate function implementations?
Unfortunately, the code being compiled is proprietary and details cannot be shared at this time.
Compiler is:
arm-none-eabi-gcc (GNU Tools for ARM Embedded Processors) 5.4.1 20160919 (release) [ARM/embedded-5-branch revision 240496]
Target is:
cortex-m3
Your help is appreciated.
--- EDIT ---
There are two files:
is the list with all the source code (source.mk):
C_FILES_SRC = $(SDK_DIR)/file001.c
C_FILES_SRC += $(SDK_DIR)/file002.c
C_FILES_SRC += $(SDK_DIR)/ .....
|
C_FILES_SRC += $(APPL_DIR)/file121.c
C_FILES_SRC += $(APPL_DIR)/file122.c
is the Makefile (short version):
include source.mk
CFLAGS = xxxxx
# create objects
%.o: %.c
$(GCC) $(CFLAGS) -MMD -MP -MF($(#:%.o=%.d) -o $# -c $<
# link it all together
executable.elf
$(GCC) $(MAIN_CFLAGS) $(LINK_SCRIPTS) -Xlinker --gc-sections $(LIBS_DIR) $(EXTRA_LINK_FLAGS) $(SPECS) -o $# $(OBJS) $(LIBS)
$(SIZE) --format=berkeley $#
In the Makefile I change nothing. Only changed the order of files in source.mk
Only static libraries (.a or .lib) exhibit "first match resolution" such that link order was critical. Since all object files are explicitly linked, you'd get a link error if there were duplicate symbols. So only the order the the .a/.lib files are presented to the linker could have that effect.
Perhaps you have two different versions of a library and are linking both in a different order?
The GNU toolchain's binutils includes the nm tool for examining symbols in object and archive (library) files.
An alternative possibility is that the executable entry point is sensitive to link order on your platform; in which case it may just be the link order of crt0 (or other run-time start-up code).
Related
I have a C library which I am using in a project. It consists of .c and .h files in src and include directories. I wrote a CMakeLists.txt file that generates a Makefile which compiles library.so.
The thing is, the library also includes .c files for tests, compatibility headers for other operating systems, and other files which I don't actually use. I would like to determine which src/header files are actually compiled into the .so library. Is there a way to do so automatically, based on CMakeLists.txt or Makefile, without going through and examining each file?
If you are using gccthen you can trick the compiler into telling you which source files it used by generating the dependency information for the next make run. If I am right, the necessary flags are:
-MT $# -MMD -MP -MF $(AUTODEP_DIR)$(notdir $#).d
This should produce for foo.cpp a file foo.o.d which contains target-prerequisites lines like:
foo.o : foo.cpp bar.hpp baz.hpp
foo.cpp :
bar.hpp :
baz.hpp :
where the line with the object file as target (the file before the :) displays what got used as C/C++ source in the compile run (all files after the :). Starting with the list of .o files which are in the libary, this should give you the exact list of files that you are looking for. It requires a bit of scripting but doesn't look too complicated.
clang and other compilers of course have their own, maybe differing set of flags but it shouldn't be too hard to pick them.
I'm creating a little static library for having thread pools, and it depends on 2 other homemade static libraries (a homemade printf and a homemade mini libc).
But sub-functions like ft_bzero are not linked in the project unless I use them on the root project, the one that needs to use thread pools library. So I have the linking error coming from my thpool lib.
Sample :
cc -Wall -Werror -Wextra -MD -I ./ -I ./jqueue -I ../libft/incs -I
../printf/incs -o .objs/thpool_create.o -c ./thpool_create.c
ar rc libthpool.a ./.objs/thpool_create.o etcetc
In the libraries, I compile every .o and use an ar rc libthpool.a *.o. Then I compile .o from main project (a single test.c actually), and then
cc .objs/test.o -o test -L./libft -L./printf -L./thpool -lft -lftprintf -lthpool -lpthread
How can I solve my errors?
Since the code in the ftpool library uses code from ft and ftprintf, you (almost certainly) need to list the libraries in the reverse order:
cc .objs/test.o -o test -L./libft -L./printf -L./thpool -lthpool -lftprintf -lft -lpthread
When scanning a static library, the linker looks for definitions of symbols that are currently undefined. If your test code only calls functions from thpool, then none of the symbols in ft are referenced when the ft library is scanned, so nothing is included from the library; if none of the symbols from ftprintf are referenced when the ftprintf library is scanned, nothing is included from ftprintf either. When it comes across the symbols in thpool that reference things from ft or ftprintf, it's too late; the linker doesn't rescan the libraries. Hence you need to list the libraries in an order such that all references from one library (A) to another (B) are found by linking (A) before (B). If the test code references some of the functions in ft or ftprintf, you may get lucky, or a bit lucky; some symbols may be linked in. But if there are functions in thpool that make the first reference to a function in ft, with the order in the question, you've lost the chance to link everything. Hence the suggested reordering.
Another (very grubby, but nonetheless effective) technique is to rescan the static libraries by listing them several times on the command line.
With shared libraries, the rules of linking are different. If a shared library satisfies any symbol, the whole library will be available, so the linker remembers all the defined symbols, and you might well get away with the original link order.
You might need to look up 'topological sort'. You should certainly aim to design your static libraries so that there are no loops in the dependencies; that leads to cycles of dependencies, and the only reliable solutions are either to rescan the libraries or combine the libraries.
I'm trying to build an application where there are two different systems interacting with each other and most of their functionalities are common. So I'm planning a directory structure like this.
Xxx
|_sys1
|
|_sys2
|
|_common
Each have a separate include and src directories.
Learned from a little browsing that keeping the compilation of common as libcmn.so will be efficient and can be linked for compiling xxx and yyy.
Though I understand how to create a .so, using them in a makefile with variables and linking header files are new to me.
Can someone guide me on how the Makefile for each should be and how linking should be done for a proper build?
In a make(1) control file, sequencing is important. Here is the general rule:
Targets placed on the same prerequisites line may be build concurrently. Order is preserved between different <code>Makefile</code> targets. So something like this:
all:: a b
a:: a.o lib/libfoo.so
${CC} ${LDFLAGS} -o $# a.o -Llib -rpath=lib -lfoo.so ${LDLIBS}
b:: b.o lib/libfoo.so
${CC} ${LDFLAGS} -o $# b.o -Llib -rpath=lib -lfoo.so ${LDLIBS}
lib/libfoo.so:
${MAKE} -C lib all
Now, on a big server you can do this:
$ make -j10 all
and everything will be built in the right order. Just be sure there is a Makefile in the lib directory that knows how to build the library.
Unless libfoo.so is significantly large, I'd make it a static libary, then you would not have to pre-program where to find it later at runtime.
To compile xxx and yyy that use your libcmn.so, they just need to include the library interface header.
To link with the library just add a -lcmn and -Lpath/to/your/lib to your LDFLAGS
To be noted that using this method, you will need to :
Add the library path to LD_LIBRARY_PATH
or add the library path to ldconfig
or move the library to a place where ldconfig looks already (/usr/lib for example)
You can also use the dlopen and dlsym functions to map your libcmn functions to function pointers at runtime. This will allow you to specify the path and name of the library to load at runtime and avoid the LD_LIBRARY_PATH problem.
You will need to add the -ldl flag to compile using this technique
I'm attempting to do a release of some software and am currently working through a script for the build process. I'm stuck on something I never thought I would be, statically linking LAPACK on x86_64 linux. During configuration AC_SEARCH_LIB([main],[lapack]) works, but compilation of the lapack units do not work, for example undefiend reference to 'dsyev_' --no lapack/blas routine goes unnoticed.
I've confirmed I have the libraries installed and even compiled them myself with the appropriate options to make them static with the same results.
Here is an example I had used in my first experience with LAPACK a few years ago that works dynamically, but not statically: http://pastebin.com/cMm3wcwF
The two methods I'm using to compile are the following,
gcc -llapack -o eigen eigen.c
gcc -static -llapack -o eigen eigen.c
Your linking order is wrong. Link libraries after the code that requires them, not before. Like this:
gcc -o eigen eigen.c -llapack
gcc -static -o eigen eigen.c -llapack
That should resolve the linkage problems.
To answer the subsequent question why this works, the GNU ld documentation say this:
It makes a difference where in the command you write this option; the
linker searches and processes libraries and object files in the order
they are specified. Thus, foo.o -lz bar.o' searches libraryz' after
file foo.o but before bar.o. If bar.o refers to functions in `z',
those functions may not be loaded.
........
Normally the files found this way are library files—archive files
whose members are object files. The linker handles an archive file by
scanning through it for members which define symbols that have so far
been referenced but not defined. But if the file that is found is an
ordinary object file, it is linked in the usual fashion.
ie. the linker is going to make one pass through a file looking for unresolved symbols, and it follows files in the order you provide them (ie. "left to right"). If you have not yet specified a dependency when a file is read, the linker will not be able to satisfy the dependency. Every object in the link list is parsed only once.
Note also that GNU ld can do reordering in cases where circular dependencies are detected when linking shared libraries or object files. But static libraries are only parsed for unknown symbols once.
I made a static library with GCC. Building of the library was OK.
When I use it the linker throws undefined reference errors on some functions. But nm says the functions are defined and exported in the static library (marked with T). I know about the linking order that I need to put the libraries after that module that needs them so this can not be a problem.
The static library was built from 3 C files. A.c B.c and D.c The D module depend on A and B (includes their headers).
No problem when I use functions from A and B but when I try to use any function from D I get undefined reference errors on them.
If I move these functions in A or B it works. But not if they are in the D module.
I'm completely run out of ideas what's going on or what is I'm overlooked.
I'm using Code::Blocks and working with plain C files.
An old trick that many times works: List each static library twice in the linking phase.
i.e., in your makefile (or whatever you're using), put:
gcc -o <outfile> <liba> <libb> <libc> <liba> <libb> <libc>
Anyway, I hope you get the idea.
I found out that I added A .cpp file to my project and I just renamed it to .c. I chose C language instead of C++ when I created the project. I did't think this could cause problems
I thought the file extension decides when the IDE chooses between gcc and g++. But not. In Code::Blocks if you add a file with a .cpp extension it will use g++. If you add a file with a .c extension it will use gcc. But if you rename the file it will use the same compiler. You have to change it explicitly in the project options.
That D module was built using g++ instead of gcc.
I realized this when I set the IDE to show me the entire command line when building not just "Compiling foo.c".
In the master make file I wrote to simplify my application/library builds, the solution I used was to run the link step twice. Using the -u linker option to specify undefined symbols on the second link.
In my make file I have a target like this:
undefined.txt:
#$(generate-undefined-syms)
which calls this macro... the first attempt at linking...
define generate-undefined-syms
$(PRINTF) "$(this_makefile): Generating undefined symbols ... \n"
$(CC) -o rubbish $(LDFLAGS) $(objects) $(LDLIBS) 2>&1 | $(GREP) 'undefined reference' > tmp.txt; \
$(SED) 's/^.*`/-Wl,-u/g' < tmp.txt > undefined.txt; \
rm -f tmp.txt rubbish
endef
As my sed/regexp skills aren't good (and I wrote this stuff in a rush) I end up with undefined.txt containing:
-uSomeSym'
-uSomeOtherSym'
i.e. with a trailing '
I then use this make syntax to strip the 's, and remove duplicates
undefined_references = $(filter-out follow, $(sort $(subst ',,$(shell cat undefined.txt))))
The 'follow' filter is because if an undefined symbol is referenced many times, a message "more references to XXX follow" appears in the output, which leads to a spurious 'follow' in the undefined.txt file e.g.
-Wl, uXXXX' follow
Finally I link the second time (note the dependency on undefined.txt)
$(application): $(library_dependencies) $(objects) undefined.txt
$(CC) -o $# $(LDFLAGS) $(undefined_references) $(objects) $(LDLIBS)
I'd totally recommed the following book by the way, as I was able to write from scratch a simple build system in a couple of days.
Managing Projects with GNU Make, Third Edition
By: Robert Mecklenburg
Perhaps you should use ranlib or the approriate ar option to provide an index to your .a file.