I'm trying to build an application where there are two different systems interacting with each other and most of their functionalities are common. So I'm planning a directory structure like this.
Xxx
|_sys1
|
|_sys2
|
|_common
Each have a separate include and src directories.
Learned from a little browsing that keeping the compilation of common as libcmn.so will be efficient and can be linked for compiling xxx and yyy.
Though I understand how to create a .so, using them in a makefile with variables and linking header files are new to me.
Can someone guide me on how the Makefile for each should be and how linking should be done for a proper build?
In a make(1) control file, sequencing is important. Here is the general rule:
Targets placed on the same prerequisites line may be build concurrently. Order is preserved between different <code>Makefile</code> targets. So something like this:
all:: a b
a:: a.o lib/libfoo.so
${CC} ${LDFLAGS} -o $# a.o -Llib -rpath=lib -lfoo.so ${LDLIBS}
b:: b.o lib/libfoo.so
${CC} ${LDFLAGS} -o $# b.o -Llib -rpath=lib -lfoo.so ${LDLIBS}
lib/libfoo.so:
${MAKE} -C lib all
Now, on a big server you can do this:
$ make -j10 all
and everything will be built in the right order. Just be sure there is a Makefile in the lib directory that knows how to build the library.
Unless libfoo.so is significantly large, I'd make it a static libary, then you would not have to pre-program where to find it later at runtime.
To compile xxx and yyy that use your libcmn.so, they just need to include the library interface header.
To link with the library just add a -lcmn and -Lpath/to/your/lib to your LDFLAGS
To be noted that using this method, you will need to :
Add the library path to LD_LIBRARY_PATH
or add the library path to ldconfig
or move the library to a place where ldconfig looks already (/usr/lib for example)
You can also use the dlopen and dlsym functions to map your libcmn functions to function pointers at runtime. This will allow you to specify the path and name of the library to load at runtime and avoid the LD_LIBRARY_PATH problem.
You will need to add the -ldl flag to compile using this technique
Related
Till now, I was using the following makefile that I have generated somehow for my school projects:
my makefile
But now I have a different situation: I am supposed to compile 4 programs for one project, while part of the code is supposed to be compiled as .so, for use for the 4 programs.
like described here:
1 - all the parts that are supposed to be compiled together as one .so file, using for example:
gcc -shared -fPIC src/file1.c src/file2.c src/file3.c -o libutils.so
3,4,5 should be compiled and linked together with this .so file, using for example:
gcc src/file4.c -L'pwd' lutils -o file4.out
the same way for all the 3 projects, and one more simple compilation of project 2.
I wandered across the net, google, your site, etc.
tried to find a solution for this situation,
without any luck.
already seen solutions like this one:
solution example
where you supply makefile with the details of the entire project structure.
I thought about dividing all the files into 4 folders, below the main folder, and creating a loop inside makefile that will compile each program in each cycle, with "if" statements to make a different compilation, according to the index. but I had no luck, it seems very complicated (maybe someone can show me a solution like that one...).
I am wondering if there is a way of making this whole compilation process generic and automatic like the current file (maybe little less),
if there is a way, I would like to study and discover it.
thank you in advance!!!
Arie
Since you have a nicely drawn tree of dependencies, you "just" need to translate this into a Makefile.
You might like to start with this:
.PHONY: all
all: reloader.exe block_finder.exe formatter.exe printdb.exe
MODULES = reloader block_finder formatter printdb linked_list bitcoin file_handler
SRCS = $(MODULES:%=%.c)
reloader.exe block_finder.exe formatter.exe printdb.exe: libbitcoin_manager.so
reloader.exe: reloader.o
block_finder.exe: block_finder.o
formatter.exe: formatter.o
printdb.exe: printdb.o
libbitcoin_manager.so: linked_list.o bitcoin.o file_handler.o
gcc -shared -fPIC $^ -o $#
%.exe: %.o
gcc $< -L. -lbitcoin_manager -o $#
%.o: %.c
gcc -c $< -o $#
%.d: %.c
gcc -MM -MT $# -MT $*.o -MF $# $<
include $(SRCS:%.c=%.d)
Because you don't have a loop in the diagram, you don't need a loop in the Makefile. Instead you put all dependent files on the left of a colon and the file they depend on on the right.
You might like to collect more "objects" in variables, for example the programs to build, the modules in the library, and so on.
I have also used a common pattern to generate the dependencies from the header files. The way shown is just one way to do it. It uses files with a ".d" extension, for "dependency." GCC has options to build these files, it scans the source and collects all included headers even if "stacked."
For example, "bitcoin.d" looks like this:
bitcoin.d bitcoin.o: bitcoin.c bitcoin.h linked_list.h definitions.h \
file_handler.h
The re-generate the dependency file on changes in the sources it is also a target, not only the object file.
EDIT:
First, using directories makes Makefiles more difficult. I don't like such structures not only for that reason, but also because they separate header files and implementation files that clearly belong to each other.
Anyway, here is an enhanced Makefile:
.PHONY: all
SRCDIR = src
INCDIR = include
BLDDIR = build
APPS = reloader block_finder formatter printdb
MODULES = reloader block_finder formatter printdb linked_list bitcoin file_handler
LIBNAME = bitcoin_manager
LIBMODULES = linked_list bitcoin file_handler
VPATH = $(SRCDIR)
SRCS = $(MODULES:%=%.c)
LIB = $(LIBNAME:%=lib%.so)
#win LIB = $(LIBNAME:%=%.lib)
EXES = $(APPS:%=%.exe)
all: $(BLDDIR) $(EXES)
$(BLDDIR):
mkdir $#
$(LIB): $(LIBMODULES:%=$(BLDDIR)/%.o)
gcc -shared -fPIC $^ -o $#
$(EXES): $(LIB)
$(EXES): %.exe: $(BLDDIR)/%.o
gcc $< -L. -l$(LIBNAME) -o $#
$(BLDDIR)/%.o: %.c
gcc -I$(INCDIR) -c $< -o $#
$(SRCDIR)/%.d: %.c
gcc -I$(INCDIR) -MM -MT $# -MT $(BLDDIR)/$*.o -MF $# $<
include $(SRCS:%.c=$(SRCDIR)/%.d)
It uses a lot more variables to simplify renaming and managing a growing library and application.
One important issue is the use of VPATH. This makes make search for sources in the list of paths assigned to it. Make sure you understand it thoroughly, search for articles and documentation. It is easy to use it wrong.
The pattern $(EXES): %.exe: $(BLDDIR)/%.o is a nice one. It consists of three parts, first a list of targets, second a generic pattern with a single target and its source. Here is means that for all executables each of them is built from its object file.
Now to your questions:
Is answered by the new proposal. I didn't add the directory but use VPATH.
Make stopped not because the exe-from-o pattern was wrong, but because it didn't find a way to build the object file needed. This is solved by the new proposal, too. To find out what happens if you delete these 4 recipes in the old proposal: you can experiment, so do it!
The dot is, like user3629249 tried to say, the present working directory. You had it in your Makefile with 'pwd' and I replaced it. This is not special to make, it is common in all major operating systems, including Windows. You might know .. which designates the parent directory.
When make starts it reads the Makefile or any given file. If this file contains include directives the files listed are checked if they need to be rebuild. make does this even if you call it with -n! After (re-)building all files to be included they are included finally. Now make has all recipes and continues with its "normal" work.
I'm working on an embedded project and I am duplicating a sample project.
Not taking the linking order of the object files into account, I just put the c files in a random order in my Makefile.
Compiling and linking yields an executable.elf of 1.9Mb.
No errors were generated but the executable didn't work.
After a long search with no solution I finally duplicated the project exactly, including the order of the c files (120 of them) and behold I got an executable.elf of 2.2Mb and no errors. AND the executable worked.
Nothing changed in compile-options and/or linking-options. Just changed the order in which the c files are listed in the makefile, and therefore the order of the object files at link time.
I suspect that there are multiple duplicate function implementations with different bodies/sizes. My hypothesis is that link time the linker, with no memory of previous link actions, just picks the first one it encounters without raising an error.
However I would like to get a hold on this provided library (all single c files, no lib *.a file) and find the duplicate function implementations. So I know in which order I should provide the c files and more importantly, why.
Two questions:
Is the description above, indeed a potential cause of the issues?
Are there other possibilities?
How to find the duplicate function implementations?
Unfortunately, the code being compiled is proprietary and details cannot be shared at this time.
Compiler is:
arm-none-eabi-gcc (GNU Tools for ARM Embedded Processors) 5.4.1 20160919 (release) [ARM/embedded-5-branch revision 240496]
Target is:
cortex-m3
Your help is appreciated.
--- EDIT ---
There are two files:
is the list with all the source code (source.mk):
C_FILES_SRC = $(SDK_DIR)/file001.c
C_FILES_SRC += $(SDK_DIR)/file002.c
C_FILES_SRC += $(SDK_DIR)/ .....
|
C_FILES_SRC += $(APPL_DIR)/file121.c
C_FILES_SRC += $(APPL_DIR)/file122.c
is the Makefile (short version):
include source.mk
CFLAGS = xxxxx
# create objects
%.o: %.c
$(GCC) $(CFLAGS) -MMD -MP -MF($(#:%.o=%.d) -o $# -c $<
# link it all together
executable.elf
$(GCC) $(MAIN_CFLAGS) $(LINK_SCRIPTS) -Xlinker --gc-sections $(LIBS_DIR) $(EXTRA_LINK_FLAGS) $(SPECS) -o $# $(OBJS) $(LIBS)
$(SIZE) --format=berkeley $#
In the Makefile I change nothing. Only changed the order of files in source.mk
Only static libraries (.a or .lib) exhibit "first match resolution" such that link order was critical. Since all object files are explicitly linked, you'd get a link error if there were duplicate symbols. So only the order the the .a/.lib files are presented to the linker could have that effect.
Perhaps you have two different versions of a library and are linking both in a different order?
The GNU toolchain's binutils includes the nm tool for examining symbols in object and archive (library) files.
An alternative possibility is that the executable entry point is sensitive to link order on your platform; in which case it may just be the link order of crt0 (or other run-time start-up code).
I have the following makefile
CXXFILES = pthreads.cpp
CXXFLAGS = -O3 -o prog -rdynamic -D_GNU_SOURCE -L./libmine
LIBS = -lpthread -ldl
all:
$(CXX) $(CXXFILES) $(LIBS) $(CXXFLAGS)
clean:
rm -f prog *.o
I am trying to include the ./libmine library within CXXFLAGS, but it seems like it is not the right way to include a static library, because when I compile the program, I get many undefined references error. So what is actually the right way to include a static library in the makefile?
use
LDFLAGS= -L<Directory where the library resides> -l<library name>
Like :
LDFLAGS = -L. -lmine
for ensuring static compilation you can also add
LDFLAGS = -static
Or you can just get rid of the whole library searching, and link with with it directly.
Say you have main.c, fun.c and a static library libmine.a.
Then you can just do in your final link line of the Makefile
$(CC) $(CFLAGS) main.o fun.o libmine.a
CXXFLAGS = -O3 -o prog -rdynamic -D_GNU_SOURCE -L./libmine
LIBS = libmine.a -lpthread
Make sure that the -L option appears ahead of the -l option; the order of options in linker command lines does matter, especially with static libraries. The -L option specifies a directory to be searched for libraries (static or shared). The -lname option specifies a library which is with libmine.a (static) or libmine.so (shared on most variants of Unix, but Mac OS X uses .dylib and HP-UX used to use .sl). Conventionally, a static library will be in a file libmine.a. This is convention, not mandatory, but if the name is not in the libmine.a format, you cannot use the -lmine notation to find it; you must list it explicitly on the compiler (linker) command line.
The -L./libmine option says "there is a sub-directory called libmine which can be searched to find libraries". I can see three possibilities:
You have such a sub-directory containing libmine.a, in which case you also need to add -lmine to the linker line (after the object files that reference the library).
You have a file libmine that is a static archive, in which case you simply list it as a file ./libmine with no -L in front.
You have a file libmine.a in the current directory that you want to pick up. You can either write ./libmine.a or -L . -lmine and both should find the library.
The -L merely gives the path where to find the .a or .so file. What you're looking for is to add -lmine to the LIBS variable.
Make that -static -lmine to force it to pick the static library (in case both static and dynamic library exist).
Addition: Suppose the path to the file has been conveyed to the linker (or compiler driver) via -L you can also specifically tell it to link libfoo.a by giving -l:libfoo.a. Note that in this case the name includes the conventional lib-prefix. You can also give a full path this way. Sometimes this is the better method to "guide" the linker to the right location.
I have written a C program that consists of a number of files, and uses APR (http://apr.apache.org) - it includes its headers and I would prefer dynamic linking of its libraries.
My problem is that currently I hard-code both the paths to APR headers as set up at my development site and the paths of the static APR library I have found.
This is what my Makefile looks like (i renamed some bits):
my_program: file1.c file2.c file3.c file4.c file5.c file6.c file7.c
#rm -f $#
$(CC) -std=c99 -pedantic -D_POSIX_C_SOURCE -I/usr/local/include/apache -L/usr/local/lib -lapr-1 $(CFLAGS) $^ -o $#
Obviously the name of the APR static library file is libapr-1.a, but I also have libapr-1.so and libapr-1.so.0 and even libapr-1.la which I am not even sure what is. So, in effect, I suspect that right now I am linking to APR statically. Aside from my goal of a respectable build system, I would like to link to APR dynamically.
Is there a common practice for me to set up an automated build that will work not only for me now, but also is flexible enough for others to build my program? I smell the proximity of the autotools, but I have absolutely zero experience with them, and would like for now to settle for the next best thing. What options do I have?
By default linker will pickup shared library (.so) and if it not found static library (.a).
So when you pass to linker -lapr-1 it will exectly what you want.
To ensure that your binrary indeed will use shared library run:
ldd your_binary
You should see apr library in output list.
I made a static library with GCC. Building of the library was OK.
When I use it the linker throws undefined reference errors on some functions. But nm says the functions are defined and exported in the static library (marked with T). I know about the linking order that I need to put the libraries after that module that needs them so this can not be a problem.
The static library was built from 3 C files. A.c B.c and D.c The D module depend on A and B (includes their headers).
No problem when I use functions from A and B but when I try to use any function from D I get undefined reference errors on them.
If I move these functions in A or B it works. But not if they are in the D module.
I'm completely run out of ideas what's going on or what is I'm overlooked.
I'm using Code::Blocks and working with plain C files.
An old trick that many times works: List each static library twice in the linking phase.
i.e., in your makefile (or whatever you're using), put:
gcc -o <outfile> <liba> <libb> <libc> <liba> <libb> <libc>
Anyway, I hope you get the idea.
I found out that I added A .cpp file to my project and I just renamed it to .c. I chose C language instead of C++ when I created the project. I did't think this could cause problems
I thought the file extension decides when the IDE chooses between gcc and g++. But not. In Code::Blocks if you add a file with a .cpp extension it will use g++. If you add a file with a .c extension it will use gcc. But if you rename the file it will use the same compiler. You have to change it explicitly in the project options.
That D module was built using g++ instead of gcc.
I realized this when I set the IDE to show me the entire command line when building not just "Compiling foo.c".
In the master make file I wrote to simplify my application/library builds, the solution I used was to run the link step twice. Using the -u linker option to specify undefined symbols on the second link.
In my make file I have a target like this:
undefined.txt:
#$(generate-undefined-syms)
which calls this macro... the first attempt at linking...
define generate-undefined-syms
$(PRINTF) "$(this_makefile): Generating undefined symbols ... \n"
$(CC) -o rubbish $(LDFLAGS) $(objects) $(LDLIBS) 2>&1 | $(GREP) 'undefined reference' > tmp.txt; \
$(SED) 's/^.*`/-Wl,-u/g' < tmp.txt > undefined.txt; \
rm -f tmp.txt rubbish
endef
As my sed/regexp skills aren't good (and I wrote this stuff in a rush) I end up with undefined.txt containing:
-uSomeSym'
-uSomeOtherSym'
i.e. with a trailing '
I then use this make syntax to strip the 's, and remove duplicates
undefined_references = $(filter-out follow, $(sort $(subst ',,$(shell cat undefined.txt))))
The 'follow' filter is because if an undefined symbol is referenced many times, a message "more references to XXX follow" appears in the output, which leads to a spurious 'follow' in the undefined.txt file e.g.
-Wl, uXXXX' follow
Finally I link the second time (note the dependency on undefined.txt)
$(application): $(library_dependencies) $(objects) undefined.txt
$(CC) -o $# $(LDFLAGS) $(undefined_references) $(objects) $(LDLIBS)
I'd totally recommed the following book by the way, as I was able to write from scratch a simple build system in a couple of days.
Managing Projects with GNU Make, Third Edition
By: Robert Mecklenburg
Perhaps you should use ranlib or the approriate ar option to provide an index to your .a file.