I've read some articles explaining how recursive makefiles were evil in the case of compiling projects with subdirectories.
But, I found this handy way to automatically generate dependencies
exec = main
objs = main.o A.o B.o C.o # main and object modules
deps = $(objs:.o=.d) # dependencies file
all: $(deps)
$(MAKE) $(exec)
-include $(deps)
%.d: %.c # how to build dependency files
$(CC) -MM $(CFLAGS) $< > $#
main: $(objs) # How to build the main exec
Building the target "all" updates the dependency files if needed, and then considers rebuilding the main exec if needed.
Is there something fundamentally flawed with this approach?
If you're using GNU make, this hack is unnecessary. GNU make is smart enough to automatically rebuild any (make)file that is included, and then restart itself:
Since the ‘.d’ files are makefiles like any others, make will remake them as necessary with no further work from you. See Remaking Makefiles.
About other make implementations, the GNU make manual has this to say:
With old make programs, it was traditional practice to use [the -M] compiler feature to generate prerequisites on demand with a command like ‘make depend’. That command would create a file depend containing all the automatically-generated prerequisites; then the makefile could use include to read them in (see Include).
As remarked by other contributors, gnu make treats .d dependencies files as Makefile and rebuilds them automagically when needed.
This makes the recursive call unnecessary, and the question falls flat.
Of course, as usual, "considered harmful" paper titles have to be read with a grain of salt, if really read at all, from a long tradition (Knuth's "rebuttal" of Dijkstra's paper cites Disjktra own fear to be believed as terribly dogmatic" and "others making a religion of it". "Fanatical advocates going over the board" !).
Related
I am taking a Coursera course and I am totally stuck in one of the project. Project demands us to create a makefile but I could not.
Here is my files and folders:
project
|---include
|---CMSIS
|---cmsis_gcc.h
|---core_cm4.h
|---core_cmFunc.h
|---core_cmInstr.h
|---ore_cmSimd.h
|------common
|---memory.h
|---platform.h
|------msp432
|---cmsis_gcc.h
|---msp_compatibility.h
|---syste_msp432.h
|---src
|---interr.c
|---main.c
|---memory.c
|---str.h
|---system_msp.h
|---makefile
|---msp432p401r.lds
|---sources.mk
Here is my sources.mk and makefile.
For now, I just want to check whether makefile succesfully find related source and include files but it does not.
I have checked several example makefiles and could not find where is the fault of mine? What I have got here is:
What I understand from error is it can not find related memory.h file. What can I do?
Any help will be appreciated.
Thanks.
Make searches for prerequisites in the working directory. It doesn't know that you have memory.h somewhere else, and will not go looking for without instructions.
One way to solve this is with the vpath directive:
vpath %.h common
You can use the same trick when you run into problems finding other prerequisites:
vpath %h common CMSIS
vpath %.c src
That will get echo "HI" to work, but you will have more trouble with a real rule that actually uses the prerequisites. To take a contrived example:
memory.o: memory.c memory.h
$(CC) -c $(CFLAGS) memory.c -o memory.o
This will fail because although Make can find the prerequisites, the compiler still doesn't know where they are.
We tell the compiler where memory.c is by referring it to the list of prerequisites:
memory.o: memory.c memory.h
$(CC) -c $(CFLAGS) $< -o memory.o
($< means the first prerequisite; Make will provide the path.) We could use the same trick (with some effort) to tell the compiler where to search for the header file, but it's usually simpler to put that in by hand:
memory.o: memory.c memory.h
$(CC) -c $(CFLAGS) -Icommon $< -o memory.o
It's important to understand that make and the compiler are two completely different programs, that do different things and work different ways. Make can run a compiler, but it can also do all sorts of other things: it could build documentation, or an entire website, or copy files, etc. Basically make is a general-purpose tool that can investigate when files are out of date and run any command you give it to make that file "up to date", whatever that may mean.
So, when you create a variable like INCLUDES = -I... that is setting a make variable to a flag that can tell the compiler where your header files are.
That means nothing to make.
Make sees this rule:
ysk: memory.h memory.c
and says "ok to build the file ysk I need to have memory.h and memory.c". It looks in the current directory and those files don't exist (because they are in src/memory.h and src/memory.c) so it fails.
You need to put the paths into your makefile, so make can find them.
Till now, I was using the following makefile that I have generated somehow for my school projects:
my makefile
But now I have a different situation: I am supposed to compile 4 programs for one project, while part of the code is supposed to be compiled as .so, for use for the 4 programs.
like described here:
1 - all the parts that are supposed to be compiled together as one .so file, using for example:
gcc -shared -fPIC src/file1.c src/file2.c src/file3.c -o libutils.so
3,4,5 should be compiled and linked together with this .so file, using for example:
gcc src/file4.c -L'pwd' lutils -o file4.out
the same way for all the 3 projects, and one more simple compilation of project 2.
I wandered across the net, google, your site, etc.
tried to find a solution for this situation,
without any luck.
already seen solutions like this one:
solution example
where you supply makefile with the details of the entire project structure.
I thought about dividing all the files into 4 folders, below the main folder, and creating a loop inside makefile that will compile each program in each cycle, with "if" statements to make a different compilation, according to the index. but I had no luck, it seems very complicated (maybe someone can show me a solution like that one...).
I am wondering if there is a way of making this whole compilation process generic and automatic like the current file (maybe little less),
if there is a way, I would like to study and discover it.
thank you in advance!!!
Arie
Since you have a nicely drawn tree of dependencies, you "just" need to translate this into a Makefile.
You might like to start with this:
.PHONY: all
all: reloader.exe block_finder.exe formatter.exe printdb.exe
MODULES = reloader block_finder formatter printdb linked_list bitcoin file_handler
SRCS = $(MODULES:%=%.c)
reloader.exe block_finder.exe formatter.exe printdb.exe: libbitcoin_manager.so
reloader.exe: reloader.o
block_finder.exe: block_finder.o
formatter.exe: formatter.o
printdb.exe: printdb.o
libbitcoin_manager.so: linked_list.o bitcoin.o file_handler.o
gcc -shared -fPIC $^ -o $#
%.exe: %.o
gcc $< -L. -lbitcoin_manager -o $#
%.o: %.c
gcc -c $< -o $#
%.d: %.c
gcc -MM -MT $# -MT $*.o -MF $# $<
include $(SRCS:%.c=%.d)
Because you don't have a loop in the diagram, you don't need a loop in the Makefile. Instead you put all dependent files on the left of a colon and the file they depend on on the right.
You might like to collect more "objects" in variables, for example the programs to build, the modules in the library, and so on.
I have also used a common pattern to generate the dependencies from the header files. The way shown is just one way to do it. It uses files with a ".d" extension, for "dependency." GCC has options to build these files, it scans the source and collects all included headers even if "stacked."
For example, "bitcoin.d" looks like this:
bitcoin.d bitcoin.o: bitcoin.c bitcoin.h linked_list.h definitions.h \
file_handler.h
The re-generate the dependency file on changes in the sources it is also a target, not only the object file.
EDIT:
First, using directories makes Makefiles more difficult. I don't like such structures not only for that reason, but also because they separate header files and implementation files that clearly belong to each other.
Anyway, here is an enhanced Makefile:
.PHONY: all
SRCDIR = src
INCDIR = include
BLDDIR = build
APPS = reloader block_finder formatter printdb
MODULES = reloader block_finder formatter printdb linked_list bitcoin file_handler
LIBNAME = bitcoin_manager
LIBMODULES = linked_list bitcoin file_handler
VPATH = $(SRCDIR)
SRCS = $(MODULES:%=%.c)
LIB = $(LIBNAME:%=lib%.so)
#win LIB = $(LIBNAME:%=%.lib)
EXES = $(APPS:%=%.exe)
all: $(BLDDIR) $(EXES)
$(BLDDIR):
mkdir $#
$(LIB): $(LIBMODULES:%=$(BLDDIR)/%.o)
gcc -shared -fPIC $^ -o $#
$(EXES): $(LIB)
$(EXES): %.exe: $(BLDDIR)/%.o
gcc $< -L. -l$(LIBNAME) -o $#
$(BLDDIR)/%.o: %.c
gcc -I$(INCDIR) -c $< -o $#
$(SRCDIR)/%.d: %.c
gcc -I$(INCDIR) -MM -MT $# -MT $(BLDDIR)/$*.o -MF $# $<
include $(SRCS:%.c=$(SRCDIR)/%.d)
It uses a lot more variables to simplify renaming and managing a growing library and application.
One important issue is the use of VPATH. This makes make search for sources in the list of paths assigned to it. Make sure you understand it thoroughly, search for articles and documentation. It is easy to use it wrong.
The pattern $(EXES): %.exe: $(BLDDIR)/%.o is a nice one. It consists of three parts, first a list of targets, second a generic pattern with a single target and its source. Here is means that for all executables each of them is built from its object file.
Now to your questions:
Is answered by the new proposal. I didn't add the directory but use VPATH.
Make stopped not because the exe-from-o pattern was wrong, but because it didn't find a way to build the object file needed. This is solved by the new proposal, too. To find out what happens if you delete these 4 recipes in the old proposal: you can experiment, so do it!
The dot is, like user3629249 tried to say, the present working directory. You had it in your Makefile with 'pwd' and I replaced it. This is not special to make, it is common in all major operating systems, including Windows. You might know .. which designates the parent directory.
When make starts it reads the Makefile or any given file. If this file contains include directives the files listed are checked if they need to be rebuild. make does this even if you call it with -n! After (re-)building all files to be included they are included finally. Now make has all recipes and continues with its "normal" work.
This is my absolute first time ever making a makefile, and I'm really trying to understand the process.
I'm trying to create a very simple makefile for a C++ project whose structure is as follows:
root folder
makefile
readme
src folder
...source files all here...
include folder
...header files for external libraries here...
lib folder
...external lib files all here...
bin folder
...output directory for built executable...
obj folder
...object files all here...
I followed the tutorial here.
Here's my makefile:
IDIR=include .
CC=g++
CFLAGS=-I$(IDIR)
ODIR=bin/obj
LDIR=lib
LIBS=none
SRC=src
_DEPS=hello.h
DEPS=$(patsubst %,$(IDIR)/,%(_DEPS))
_OBJ=file1.o file2.o
OBJ=$(patsubst %,$(ODIR)/%,$(_OBJ))
$(ODIR)/%.o: $(SRC)/%.cpp $(DEPS)
$(CC) -c -o $# $< $(CFLAGS) # $(LIBS)
test_proj: $(OBJ)
$(CC) -o $# $^ $(CFLAGS)
.PHONY: clean
clean:
rm -f $(ODIR)/*.o *~ core $(INCDIR)/*~
When I run make on this, I get the following error:
g++ -o .o
g++: fatal error: no input files
compilation terminated.
<builtin>: recipe for target '.o' failed
mingw32-make.exe: *** [.o] Error 1
I'm using GNU Make 3.82.90 built for i686-pc-mingw32, if that matters at all.
Can anyone point out whatever ridiculous error I'm making?
IDIR=include .
is the first problem. Replace it by:
IDIR=include
With your code CFLAGS is expanded as:
-Iinclude .
It does not make sense, I'm afraid. The second problem is:
DEPS=$(patsubst %,$(IDIR)/,%(_DEPS))
which should probably be:
DEPS=$(patsubst %,$(IDIR)/%,$(_DEPS))
and would expand as:
DEPS=include/hello.h
if you fix the first problem, else as:
DEPS=include ./hello.h
which does not make sense neither. The cumulated effect of these two errors are strange recipes (I didn't try to expand them by hand) that probably trigger a make implicit rule with wrong parameters.
IDIR=include .
CC=g++
CFLAGS=-I$(IDIR)
This is wrong. First, for C++ code, use CXX not CC and CXXFLAGS not CFLAGS. Run make -p to understand the builtin rules of your make.
Then -I$(IDIR) does not "distribute" the -I, and IDIR is never used elsewhere. So I suggest to start your Makefile with:
CXX=g++
MY_CXX_LANG_FLAGS= -std=c++11
MY_CXX_WARN_FLAGS= -Wall -Wextra
MY_CXX_INCL_FLAGS= -I. -Iinclude
MY_CXX_MACRO_FLAGS= -DMYFOO=32
### replace with -O2 for a release build below
MY_CXX_OPTIM_FLAGS= -g
CXXFLAGS= $(MY_CXX_LANG_FLAGS) $(MY_CXX_WARN_FLAGS) \
$(MY_CXX_INCL_FLAGS) $(MY_CXX_MACRO_FLAGS)
I won't improve your Makefile, but I do suggest to upgrade to GNU make version 4 if possible (and compiling make 4.1 from its source code is worthwhile in 2015) for that purpose. If possible enable GUILE scripting in it.
If you are forced to use make 3.82 debug your Makefile using remake (with -x); if you can afford a make version 4 use its --trace option
BTW, you might consider using automatic dependencies, that is generating dependencies by passing -M or -MG (etc) flags of g++, see that.
At last, a simple project for a small program (less than a hundred thousands of source lines) might just put all (a few dozens of) its files in the current directory (then the Makefile could be simpler); your proposed directory structure might be arcane for a simple project (but could worth the pain if you have millions of C++ source lines of code). I've given several simple examples of Makefile, e.g. this & that. And GNU make source code itself has a less complex file tree that what you want.
BTW, I strongly disagree with the opinions of that answer (which I did upvote, since it is helpful). I don't feel that GNU make is senile, but I regret that, instead of using recent features available on recent versions (4.x) of make, many people prefer to use complex and arcane Makefile generators (like cmake) instead of coding a clever Makefile (for make version 4 specifically).
At last, you could use other builders, e.g. omake, icmake, ....
I have the following Makefile.am which is supposed to create foo.hdb and foo.cdb from foo.h (via the Python script):
TESTS = check_foo
check_PROGRAMS = check_foo
check_foo_SOURCES = check_foo.c $(top_builddir)/src/isti.h \
foo.cdb foo.h foo.hdb
check_foo_CFLAGS = #CHECK_CFLAGS# $(all_includes) -I../../clib/src/
check_foo_LDADD = $(top_builddir)/src/libcorm.la #CHECK_LIBS# -lsqlite3
%.hdb %.cdb: %.h
PYTHONPATH=$(top_builddir)/cgen/src python $(top_builddir)/cgen/src/isti/cgen/run.py $<
clean-local:
rm -f *.hdb *.cdb
However, although make foo.hdb and make foo.cdb work (call the Python code and generates the foo.hdb and foo.cdb files from foo.h), make clean check (or the two separately) does not (missing foo.hdb - no such file) - the pattern rule is not called to generate foo.hdb from foo.h.
In other words: the pattern rule is not being called for the files listed in check_foo_SOURCES.
How can I make this work? The rest of the autotools infrastructure is working fine. From looking at the Makefile I suspect the issue is with how autotools expands the check sources.
This is all on Linux with Gnu make. Here is the Makefile.
[Updated slightly to reflect the help from MadScientist].
Later update
The following Makefile (just make, not autotools) works fine, so the issue seems to be related to autotools and check support.
all: check_foo
CFLAGS=-I../../clib/src
LDFLAGS=-L../../clib/src/.libs
check_foo: check_foo.c foo.h corm_foo.h corm_foo.c
gcc $(CFLAGS) $(LDFLAGS) $^ -o $# -lcorm -lsqlite3
corm_%.h corm_%.c: %.h
PYTHONPATH=../../cgen/src python ../../cgen/src/isti/cgen/run.py $<
clean:
rm -f corm_*.h corm_*.c
rm -f *.o
(Note that I've switched from xxx.hdb to corm_xxx.h, etc, so that file extensions remain OK).
More Details
Since it seems to be related to the CHECK macros, this is configure.ac:
AC_INIT([corm], [0.1], [a.cooke#isti.com])
AC_CONFIG_MACRO_DIR([m4])
PKG_CHECK_MODULES([CHECK], [check >= 0.9.4])
AM_INIT_AUTOMAKE([-Wall foreign -Werror])
AC_PROG_CC_C99
AM_PROG_CC_C_O
LT_INIT
AC_CONFIG_HEADERS([config.h])
AC_CONFIG_FILES([Makefile clib/Makefile clib/src/Makefile clib/tests/Makefile clib/docs/Makefile cgen/Makefile cgen/docs/Makefile example/Makefile example/src/Makefile])
AC_CHECK_PROGS([DOXYGEN], [doxygen], AC_MSG_WARN([Doxygen not found - continuing without Doxygen support]))
AM_CONDITIONAL([HAVE_DOXYGEN], [test -n "$DOXYGEN"])
AM_COND_IF([HAVE_DOXYGEN], [AC_CONFIG_FILES([clib/docs/Doxyfile cgen/docs/Doxyfile])])
SOLUTION
OK, so summarizing the various things below, there were two important issues (once I had fixed file extensions - see the "plain" makefile and fceller's answer), either one of which was sufficient to make things work:
(The handling of) Header files is complicated. Because of auto-dependencies, programatically generated header files break things. The solution is to use BUILT_SOURCES
But (the handling of) .c files is not complicated. So putting the corm_foo.c in front of check_foo.c would trigger the generation of that file. Since that also generates corm_foo.h, everything works (because check_foo.c now can include corm_foo.h).
Also, fceller has some good general points about tidy makefiles and explains why the "plain" makefile works.
The line
%.cdb: %.hdb
does not do anything. Check the GNU make manual and you'll see that a pattern rule without a command line is used to DELETE a previously defined rule with that same pattern. Since there's no previous rule, this is essentially a no-op.
If you have a command that creates TWO output files with ONE invocation of a rule, then you need to put both patterns in the same rule, like this:
%.cdb %.hdb: %.h
PYTHONPATH=$(top_builddir)/cgen/src python $(top_builddir)/cgen/src/isti/cgen/run.py $<
This will tell GNU make that both targets are generated from one invocation of the rule. BE AWARE! This syntax only has this behavior for pattern rules. Explicit rules with multiple targets do something entirely different (confusingly enough).
As for the make clean behavior, I'm not sure. The makefile generated by automake is too complex for me to just read it; it would require some debugging (run it and trace what happens). However, I suspect that the rules there are not designed properly to allow a clean target and a build target to both be run in the same invocation of make. Run the two commands separately and see if that works better:
make clean
make check
You need to tell automake that foo.hdb is a source file that is to be constructed. Add the following to Makefile.am:
BUILT_SOURCES = foo.hdb
First of all: you do not need to include the "*.h" in *_SOURCES. The automake will generated code to generate the dependencies. From the manual: Header files listed in a _SOURCES definition will be included in the distribution but otherwise ignored
The change you made in the plain Makefile ("Note that I've switched from xxx.hdb to corm_xxx.h, etc, so that file extensions remain OK") is essential. The automake FILTERS the *_SOURCES list using the filename extension in order to see what to call (CC, CXX, F77).
The following Makefile.am will work:
TEST = check_foo
check_PROGRAMS = check_foo
check_foo_SOURCES = check_foo.c foo.db.c
check_foo_CFLAGS =
check_foo_LDADD =
%.db.c %.db.h: %.h
echo "int i = 1;" > foo.db.c
echo "int j;" > foo.db.h
clean-local:
rm -f *.db.h *.db.c
Let me try to add an indirect answer/discussion to the existing direct ones.
I recently moved away from make for the same kind of reasons you experienced: it is extremely powerful but sometimes a bit difficult to debug when things do not go as expected.
I recently discovered pydoit which is very promising as a debuggable replacement build tool for make. Since the notion of "pattern rules" was not present in it, I proposed an independent package to do the job: fprules.
This is how you would perform the same task that you mention in your post, with doit and fprules:
from fprules import file_pattern
# all: check_foo
DOIT_CONFIG = {'default_tasks': ['check_foo']}
CFLAGS = '-I../../clib/src'
LDFLAGS = '-L../../clib/src/.libs'
# check_foo: check_foo.c foo.h corm_foo.h corm_foo.c
# gcc $(CFLAGS) $(LDFLAGS) $^ -o $# -lcorm -lsqlite3
def task_check_foo():
"""
Compiles the `check_foo` executable
"""
src_files = ('check_foo.c', 'foo.h', 'corm_foo.h', 'corm_foo.c')
dst_file = 'check_foo'
return {
'file_dep': src_files,
'actions': ["gcc %s %s %s -o %s -lcorm -lsqlite3" % (CFLAGS, LDFLAGS, ' '.join(src_files), dst_file)],
'verbosity': 2,
'targets': [dst_file],
'clean': True
}
# corm_%.h corm_%.c: %.h
# PYTHONPATH=../../cgen/src python ../../cgen/src/isti/cgen/run.py $<
def task_gen_corm():
"""
Generates files `corm_%.h` and `corm_%.c`
for each header file `%.h`.
"""
for data in file_pattern('./*.h', dict(h_file='./corm_%.h', c_file='./corm_%.c')):
yield {
'name': data.name,
'file_dep': [data.src_path],
'actions': ["PYTHONPATH=../../cgen/src python ../../cgen/src/isti/cgen/run.py %s" % data.src_path],
'verbosity': 2,
'targets': [data.h_file, data.c_file],
'clean': True
}
# clean:
# rm -f corm_*.h corm_*.c
# rm -f *.o
# No need to create tasks for this:
# with 'clean': True, `doit clean` will clean all target files
Then simply run doit in the folder.
Do not hesitate to provide feedback on the projects pages if needed: for example multiline commands are not supported, you can vote for them if you too feel that they are missing: https://github.com/pydoit/doit/issues/314
I have a Linux GNU C project that requires building output for two different hardware devices, using a common C source code base, but different makefiles. Presently I have two makefiles in the same folder, one for each device, and when I make a change to the code, I have to first do "make clean" to make the first model, then "make clean" to make the second model. This is because they use different compilers and processors. Even if a code file didn't change, I have to recompile it for the other device.
What I would like to do is use a different folder for the second model, so it stores a separate copy of *.d and *.o files. I would not have to "make clean", only recompile the changed sources. I have looked at makefile syntax, and being new to Linux, can only scratch my head at the cryptic nature of this stuff.
One method I'm considering would update the .c & .h files from model_1 folder into model_2 folder. Can someone provide me with a simple makefile that will copy only newer *.c and *.h files from one folder to another?
Alternatively, there must be a way to have a common source folder, and separate output folders, so duplicated source files are not required. Any help to achieve that is appreciated. I can provide the makefiles if you want to look at them.
You want to generated files (object and dependencies) put into a separate folder for each build type as it compiles. Here's what I had do that may work for you:
$(PRODUCT1_OBJDIR)/%.o $(PRODUCT1_OBJDIR)/%.d: %.cpp
#mkdir -p $(#D)
$(CXX) $(PRODUCT1_DEPSFLAGS) $(CXXFLAGS) $(INCLUDE_DIR) $< -o $(PRODUCT1_OBJDIR)/$*.o
$(PRODUCT2_OBJDIR)/%.o $(PRODUCT2_OBJDIR)/%.d: %.cpp
#mkdir -p $(#D)
$(CXX) $(PRODUCT2_DEPSFLAGS) $(CXXFLAGS) $(INCLUDE_DIR) $< -o $(PRODUCT2_OBJDIR)/$*.o
$PRODUCT1_OBJDIR and $PRODUCT2_OBJDIR are variables names for the directory where you wish to have the generated files stored. This will check for changes to dependencies and recompile if needed.
If you still have problems, get back with feedback, will try and sort you out.
You could compile your source files to object files in different directories ("folder" is not really the appropriate word on Unix). You just have to set appropriate make rules. And you might use other builders like omake, scons, ...
You could use remake to debug your GNU Makefile-s. You could have inside a rule like
$(OBJDIR)/%.o: $(SRCDIR)/%.c
$(COMPILE.c) -c $^ -o $#
And you could set (earlier in your Makefile) variables with e.g.
OBJDIR:=obj-$(shell uname -m)
or something better
I do suggest to read GNU make's manual; it has an introduction to makefiles.
This can be easily achieved with makepp's repository mechanism. If you call makepp -R../src ARCH=whatever then it will symbolically link all parts of ../src under the current directory and compile here.
You can even create a file .makepprc and put in any options specific to this architecture, so you'll never get confused which command to call where.
If your different architectures have identically produced files (like generated sources), you can even build in ../src and the other architecture will pick up everything that doesn't depend on your current compile options.
There is much more to makepp. Besides doing almost all that GNU make can, there are lots more useful things, and you can even extend your makefiles with some Perl programming.