I have the following Makefile.am which is supposed to create foo.hdb and foo.cdb from foo.h (via the Python script):
TESTS = check_foo
check_PROGRAMS = check_foo
check_foo_SOURCES = check_foo.c $(top_builddir)/src/isti.h \
foo.cdb foo.h foo.hdb
check_foo_CFLAGS = #CHECK_CFLAGS# $(all_includes) -I../../clib/src/
check_foo_LDADD = $(top_builddir)/src/libcorm.la #CHECK_LIBS# -lsqlite3
%.hdb %.cdb: %.h
PYTHONPATH=$(top_builddir)/cgen/src python $(top_builddir)/cgen/src/isti/cgen/run.py $<
clean-local:
rm -f *.hdb *.cdb
However, although make foo.hdb and make foo.cdb work (call the Python code and generates the foo.hdb and foo.cdb files from foo.h), make clean check (or the two separately) does not (missing foo.hdb - no such file) - the pattern rule is not called to generate foo.hdb from foo.h.
In other words: the pattern rule is not being called for the files listed in check_foo_SOURCES.
How can I make this work? The rest of the autotools infrastructure is working fine. From looking at the Makefile I suspect the issue is with how autotools expands the check sources.
This is all on Linux with Gnu make. Here is the Makefile.
[Updated slightly to reflect the help from MadScientist].
Later update
The following Makefile (just make, not autotools) works fine, so the issue seems to be related to autotools and check support.
all: check_foo
CFLAGS=-I../../clib/src
LDFLAGS=-L../../clib/src/.libs
check_foo: check_foo.c foo.h corm_foo.h corm_foo.c
gcc $(CFLAGS) $(LDFLAGS) $^ -o $# -lcorm -lsqlite3
corm_%.h corm_%.c: %.h
PYTHONPATH=../../cgen/src python ../../cgen/src/isti/cgen/run.py $<
clean:
rm -f corm_*.h corm_*.c
rm -f *.o
(Note that I've switched from xxx.hdb to corm_xxx.h, etc, so that file extensions remain OK).
More Details
Since it seems to be related to the CHECK macros, this is configure.ac:
AC_INIT([corm], [0.1], [a.cooke#isti.com])
AC_CONFIG_MACRO_DIR([m4])
PKG_CHECK_MODULES([CHECK], [check >= 0.9.4])
AM_INIT_AUTOMAKE([-Wall foreign -Werror])
AC_PROG_CC_C99
AM_PROG_CC_C_O
LT_INIT
AC_CONFIG_HEADERS([config.h])
AC_CONFIG_FILES([Makefile clib/Makefile clib/src/Makefile clib/tests/Makefile clib/docs/Makefile cgen/Makefile cgen/docs/Makefile example/Makefile example/src/Makefile])
AC_CHECK_PROGS([DOXYGEN], [doxygen], AC_MSG_WARN([Doxygen not found - continuing without Doxygen support]))
AM_CONDITIONAL([HAVE_DOXYGEN], [test -n "$DOXYGEN"])
AM_COND_IF([HAVE_DOXYGEN], [AC_CONFIG_FILES([clib/docs/Doxyfile cgen/docs/Doxyfile])])
SOLUTION
OK, so summarizing the various things below, there were two important issues (once I had fixed file extensions - see the "plain" makefile and fceller's answer), either one of which was sufficient to make things work:
(The handling of) Header files is complicated. Because of auto-dependencies, programatically generated header files break things. The solution is to use BUILT_SOURCES
But (the handling of) .c files is not complicated. So putting the corm_foo.c in front of check_foo.c would trigger the generation of that file. Since that also generates corm_foo.h, everything works (because check_foo.c now can include corm_foo.h).
Also, fceller has some good general points about tidy makefiles and explains why the "plain" makefile works.
The line
%.cdb: %.hdb
does not do anything. Check the GNU make manual and you'll see that a pattern rule without a command line is used to DELETE a previously defined rule with that same pattern. Since there's no previous rule, this is essentially a no-op.
If you have a command that creates TWO output files with ONE invocation of a rule, then you need to put both patterns in the same rule, like this:
%.cdb %.hdb: %.h
PYTHONPATH=$(top_builddir)/cgen/src python $(top_builddir)/cgen/src/isti/cgen/run.py $<
This will tell GNU make that both targets are generated from one invocation of the rule. BE AWARE! This syntax only has this behavior for pattern rules. Explicit rules with multiple targets do something entirely different (confusingly enough).
As for the make clean behavior, I'm not sure. The makefile generated by automake is too complex for me to just read it; it would require some debugging (run it and trace what happens). However, I suspect that the rules there are not designed properly to allow a clean target and a build target to both be run in the same invocation of make. Run the two commands separately and see if that works better:
make clean
make check
You need to tell automake that foo.hdb is a source file that is to be constructed. Add the following to Makefile.am:
BUILT_SOURCES = foo.hdb
First of all: you do not need to include the "*.h" in *_SOURCES. The automake will generated code to generate the dependencies. From the manual: Header files listed in a _SOURCES definition will be included in the distribution but otherwise ignored
The change you made in the plain Makefile ("Note that I've switched from xxx.hdb to corm_xxx.h, etc, so that file extensions remain OK") is essential. The automake FILTERS the *_SOURCES list using the filename extension in order to see what to call (CC, CXX, F77).
The following Makefile.am will work:
TEST = check_foo
check_PROGRAMS = check_foo
check_foo_SOURCES = check_foo.c foo.db.c
check_foo_CFLAGS =
check_foo_LDADD =
%.db.c %.db.h: %.h
echo "int i = 1;" > foo.db.c
echo "int j;" > foo.db.h
clean-local:
rm -f *.db.h *.db.c
Let me try to add an indirect answer/discussion to the existing direct ones.
I recently moved away from make for the same kind of reasons you experienced: it is extremely powerful but sometimes a bit difficult to debug when things do not go as expected.
I recently discovered pydoit which is very promising as a debuggable replacement build tool for make. Since the notion of "pattern rules" was not present in it, I proposed an independent package to do the job: fprules.
This is how you would perform the same task that you mention in your post, with doit and fprules:
from fprules import file_pattern
# all: check_foo
DOIT_CONFIG = {'default_tasks': ['check_foo']}
CFLAGS = '-I../../clib/src'
LDFLAGS = '-L../../clib/src/.libs'
# check_foo: check_foo.c foo.h corm_foo.h corm_foo.c
# gcc $(CFLAGS) $(LDFLAGS) $^ -o $# -lcorm -lsqlite3
def task_check_foo():
"""
Compiles the `check_foo` executable
"""
src_files = ('check_foo.c', 'foo.h', 'corm_foo.h', 'corm_foo.c')
dst_file = 'check_foo'
return {
'file_dep': src_files,
'actions': ["gcc %s %s %s -o %s -lcorm -lsqlite3" % (CFLAGS, LDFLAGS, ' '.join(src_files), dst_file)],
'verbosity': 2,
'targets': [dst_file],
'clean': True
}
# corm_%.h corm_%.c: %.h
# PYTHONPATH=../../cgen/src python ../../cgen/src/isti/cgen/run.py $<
def task_gen_corm():
"""
Generates files `corm_%.h` and `corm_%.c`
for each header file `%.h`.
"""
for data in file_pattern('./*.h', dict(h_file='./corm_%.h', c_file='./corm_%.c')):
yield {
'name': data.name,
'file_dep': [data.src_path],
'actions': ["PYTHONPATH=../../cgen/src python ../../cgen/src/isti/cgen/run.py %s" % data.src_path],
'verbosity': 2,
'targets': [data.h_file, data.c_file],
'clean': True
}
# clean:
# rm -f corm_*.h corm_*.c
# rm -f *.o
# No need to create tasks for this:
# with 'clean': True, `doit clean` will clean all target files
Then simply run doit in the folder.
Do not hesitate to provide feedback on the projects pages if needed: for example multiline commands are not supported, you can vote for them if you too feel that they are missing: https://github.com/pydoit/doit/issues/314
Related
I've read some articles explaining how recursive makefiles were evil in the case of compiling projects with subdirectories.
But, I found this handy way to automatically generate dependencies
exec = main
objs = main.o A.o B.o C.o # main and object modules
deps = $(objs:.o=.d) # dependencies file
all: $(deps)
$(MAKE) $(exec)
-include $(deps)
%.d: %.c # how to build dependency files
$(CC) -MM $(CFLAGS) $< > $#
main: $(objs) # How to build the main exec
Building the target "all" updates the dependency files if needed, and then considers rebuilding the main exec if needed.
Is there something fundamentally flawed with this approach?
If you're using GNU make, this hack is unnecessary. GNU make is smart enough to automatically rebuild any (make)file that is included, and then restart itself:
Since the ‘.d’ files are makefiles like any others, make will remake them as necessary with no further work from you. See Remaking Makefiles.
About other make implementations, the GNU make manual has this to say:
With old make programs, it was traditional practice to use [the -M] compiler feature to generate prerequisites on demand with a command like ‘make depend’. That command would create a file depend containing all the automatically-generated prerequisites; then the makefile could use include to read them in (see Include).
As remarked by other contributors, gnu make treats .d dependencies files as Makefile and rebuilds them automagically when needed.
This makes the recursive call unnecessary, and the question falls flat.
Of course, as usual, "considered harmful" paper titles have to be read with a grain of salt, if really read at all, from a long tradition (Knuth's "rebuttal" of Dijkstra's paper cites Disjktra own fear to be believed as terribly dogmatic" and "others making a religion of it". "Fanatical advocates going over the board" !).
This is my absolute first time ever making a makefile, and I'm really trying to understand the process.
I'm trying to create a very simple makefile for a C++ project whose structure is as follows:
root folder
makefile
readme
src folder
...source files all here...
include folder
...header files for external libraries here...
lib folder
...external lib files all here...
bin folder
...output directory for built executable...
obj folder
...object files all here...
I followed the tutorial here.
Here's my makefile:
IDIR=include .
CC=g++
CFLAGS=-I$(IDIR)
ODIR=bin/obj
LDIR=lib
LIBS=none
SRC=src
_DEPS=hello.h
DEPS=$(patsubst %,$(IDIR)/,%(_DEPS))
_OBJ=file1.o file2.o
OBJ=$(patsubst %,$(ODIR)/%,$(_OBJ))
$(ODIR)/%.o: $(SRC)/%.cpp $(DEPS)
$(CC) -c -o $# $< $(CFLAGS) # $(LIBS)
test_proj: $(OBJ)
$(CC) -o $# $^ $(CFLAGS)
.PHONY: clean
clean:
rm -f $(ODIR)/*.o *~ core $(INCDIR)/*~
When I run make on this, I get the following error:
g++ -o .o
g++: fatal error: no input files
compilation terminated.
<builtin>: recipe for target '.o' failed
mingw32-make.exe: *** [.o] Error 1
I'm using GNU Make 3.82.90 built for i686-pc-mingw32, if that matters at all.
Can anyone point out whatever ridiculous error I'm making?
IDIR=include .
is the first problem. Replace it by:
IDIR=include
With your code CFLAGS is expanded as:
-Iinclude .
It does not make sense, I'm afraid. The second problem is:
DEPS=$(patsubst %,$(IDIR)/,%(_DEPS))
which should probably be:
DEPS=$(patsubst %,$(IDIR)/%,$(_DEPS))
and would expand as:
DEPS=include/hello.h
if you fix the first problem, else as:
DEPS=include ./hello.h
which does not make sense neither. The cumulated effect of these two errors are strange recipes (I didn't try to expand them by hand) that probably trigger a make implicit rule with wrong parameters.
IDIR=include .
CC=g++
CFLAGS=-I$(IDIR)
This is wrong. First, for C++ code, use CXX not CC and CXXFLAGS not CFLAGS. Run make -p to understand the builtin rules of your make.
Then -I$(IDIR) does not "distribute" the -I, and IDIR is never used elsewhere. So I suggest to start your Makefile with:
CXX=g++
MY_CXX_LANG_FLAGS= -std=c++11
MY_CXX_WARN_FLAGS= -Wall -Wextra
MY_CXX_INCL_FLAGS= -I. -Iinclude
MY_CXX_MACRO_FLAGS= -DMYFOO=32
### replace with -O2 for a release build below
MY_CXX_OPTIM_FLAGS= -g
CXXFLAGS= $(MY_CXX_LANG_FLAGS) $(MY_CXX_WARN_FLAGS) \
$(MY_CXX_INCL_FLAGS) $(MY_CXX_MACRO_FLAGS)
I won't improve your Makefile, but I do suggest to upgrade to GNU make version 4 if possible (and compiling make 4.1 from its source code is worthwhile in 2015) for that purpose. If possible enable GUILE scripting in it.
If you are forced to use make 3.82 debug your Makefile using remake (with -x); if you can afford a make version 4 use its --trace option
BTW, you might consider using automatic dependencies, that is generating dependencies by passing -M or -MG (etc) flags of g++, see that.
At last, a simple project for a small program (less than a hundred thousands of source lines) might just put all (a few dozens of) its files in the current directory (then the Makefile could be simpler); your proposed directory structure might be arcane for a simple project (but could worth the pain if you have millions of C++ source lines of code). I've given several simple examples of Makefile, e.g. this & that. And GNU make source code itself has a less complex file tree that what you want.
BTW, I strongly disagree with the opinions of that answer (which I did upvote, since it is helpful). I don't feel that GNU make is senile, but I regret that, instead of using recent features available on recent versions (4.x) of make, many people prefer to use complex and arcane Makefile generators (like cmake) instead of coding a clever Makefile (for make version 4 specifically).
At last, you could use other builders, e.g. omake, icmake, ....
I was teaching myself GNU Make and thought a look at the Redis Makefile would teach me a thing or two about the tool.
The rule that compiles the source file to the object file is here:
%.o: %.c .make-prerequisites
$(REDIS_CC) -c $<
Notice that the suffix rule just mentions the C source file (with %.c) as a prerequisite.
But if I add a echo in the middle and run make:
%.o: %.c .make-prerequisites
echo $^
$(REDIS_CC) -c $<
Then the first few lines of the output from make is like below:
cd src && make all
make[1]: Entering directory `/home/cltpadmin/code/redis/src'
echo adlist.c .make-prerequisites adlist.h zmalloc.h
adlist.c .make-prerequisites adlist.h zmalloc.h
CC adlist.o
How did make know that adlist.c depends on adlist.h and zmalloc.h?
The prerequisites in question come from line one of the Makefile.dep included makefile (included on line 134).
The dep target on line 136 generates that file.
This was a fairly common (though entirely avoidable) step for using the compiler to generate the necessary header file includes. This static method also has issues with conditional header includes I believe.
To clarify, the "avoidable" part of this is that it need not be a separate step and a static dependency file at all. See Advanced Auto-Dependency Generation for details about this idea.
In my current project, more than 200MB C files are present. While making targets I can able to see only < 100MB is used to create a binary. How, to identify what are the files are not touched while creating targets. So, that we can strip it off and make the codebase more sleek.
How, to identify the list of files created for target while compiling?
I think the best approach is to use one of the methods Linux offers for monitoring file system access:
inotifywait
kfsmd
loggerfs
auditd / auditctl / ausearch
You could also run the make command with strace (strace -f make) and parse the output for open() calls.
That is what I used to do in the past.
Take your pick.
I would probably:
make clean
make 2>out to capture the full log of the build to a file
grepthe out file for gcc invocations
Filter out the filenames from the gcc invocations
Build a sorted list of those filenames
Build a sorted list of filenames in the project.
Compare the two.
Or something like that. Shouldn't be too hard.
Another method is to open the (autogenerated) dependency files for you executables and shared libraries. They are going to contain .o as prerequisites, which normally correspond to .c.
This works best for non-recursive make build systems with complete dependency trees.
If your make is GNU make, as seems likely, you could consider
variations on the following approach. (For simplicity I'll assume that the only
source files of interest are .c files.)
Say we have Makefile in directory dir that makes some default
target and possibly others, including intermediate targets.
You can use another makefile, say, unused_srcs.mk that you include in
Makefile, such that in dir you can run:
make unused_srcs
which will display a list of all the source files in dir that don't
contribute to the default target. Or run:
make unused_srcs target=TARG
for some target TARG, and get a list of all the source files in dir that
don't contribute to TARG.
This assumes that Makefile has a clean target, or maybe several
variants of clean[-???], that can suitably clean the default target or
any target you might be interested in, and that you don't mind
a suitable clean[-???] being invoked for the unused_srcs target.
For example, in dir we have:
main.c
extern void boo(void);
int main(void)
{
boo();
return 0;
}
boo.c
#include <stdio.h>
void boo(void)
{
puts("Boo!");
}
noop.c
void noop(void){}
Makefile
.phony: all clean
all: boo
objs = main.o boo.o
boo: $(objs)
gcc -o $# $^
clean-obj:
rm -f *.o
clean: clean-obj
rm -f boo
# include /some/standard/place/unused_srcs.mk
include unused_srcs.mk
And in some standard place which, for simplicity, we'll assume is again dir,
we have:
unused_srcs.mk
.phony: unused_srcs
ifndef $(target)
target = all
endif
ifndef $(cleaner)
cleaner = clean
endif
all_srcs = $(wildcard *.c)
used_srcs = $(filter %.c,$(shell ($(MAKE) $(cleaner) && $(MAKE) -n -d $(target)) \
| grep -e 'Considering target file' | sed -e "s/[\`\']//g" -e "s/\.$$//g" -))
unused_srcs = $(filter-out $(used_srcs),$(all_srcs))
unused_srcs:
#echo $(unused_srcs)
In this project:-
noop.c is unused by boo (it is completely superfluous)
noop.c and main.c are unused by boo.o
noop.c and boo.c are unused by main.o
And we can discover such facts with make unused_srcs commands:
$ make unused_srcs
noop.c
$ make unused_srcs target=boo
noop.c
$ make unused_srcs target=boo.o
main.c noop.c
$ make unused_srcs cleaner=clean-obj target=boo.o
main.c noop.c
$ make unused_srcs target=main.o
boo.c noop.c
$ make unused_srcs target=clean
boo.c main.c noop.c
Clearly all the heavy-lifting is done in the evaluation of $(used_srcs)
and $(unused_srcs) in unused_srcs.mk, and mostly the former.
The evaluation of $(used_srcs) exploits GNU make's -d option, which
generates fulsome debugging info, together with the fact that any source
file that does contribute to the chosen TARG must itself be considered
a potential target when TARG is made.
Together, these things mean that if we clean TARG and then run make -n -d TARG
the output will be the debugging info of a dry-run (-n) of make TARG,
and in this output a line like:
Considering target file `FILE.c'.
will appear for each .c file FILE.c that contributes to TARG, and
for no other .c files that may be present.
The evaluation of $(used_srcs) generates that debugging info and filters
it to extract the names of the .c files.
The evaluating of $(unused_srcs) then just weeds out the $(used_srcs)
from the list of all the .c files there are.
I have implemented a binary tree program which includes the tree.c with the functions, the tree.h with the declarations of them and a main.c for testing.
Also, I have a makefile which is:
CC=gcc
CFLAGS=-g -Wall
DEPS = tree.h
OBJ = main.o tree.o
%.o: %.c $(DEPS)
$(CC) -c -o $# $< $(CFLAGS)
tree: $(OBJ)
$(CC) -o $# $^ $(CFLAGS)
clean:
rm -f *.o tree
Now I want to make it generate a library not only an object file for the binary trees functions and afterwards to generate the documentation of doxygen inside the makefile. Any help would be helpful.
I know that my answer comes in a bit late, but i hope someone will benefit from this.
I have a makefile that generates Doxygen doc.
You have to twist Doxygen a tiny bit
Create the Doxygen setup file that fits Your need, then open that in an editor and remove the lines containg the following two settings (they will be added by the make file later)
INPUT
FILE_PATTERNS
add this line
#INCLUDE = doxyfile.inc
Save this file under a different name I use Doxyfile.mk
in You makefile You need a list of sources and the directories where they are located
example
SRCS = $(OBJS:.o=.c)
SRCDIRS = ./src
SRCDIRS += ./other_src
Now You can put this rule in the Makefile, it will create the file doxyfile.inc that contains the settings You removed from Doxyfile.mk.
.PHONY: all clean distclean doxy
# If makefile changes, maybe the list of sources has changed, so update doxygens list
doxyfile.inc: Makefile.mk
echo INPUT = $(SRCDIRS) > doxyfile.inc
echo FILE_PATTERNS = *.h $(SRCS) >> doxyfile.inc
doxy: doxyfile.inc $(SRCS)
doxygen.exe doxyfile.mk
Bonus: If run from inside an IDE like Eclipse the errors that Doxygen spits out becomes clickable and will jump to the bad comment.
Well, I don't really know the syntax for the doxygen command, so I'll make a generic answer:
in your Makefile, each
term: [dep]
action
is a target.
So if you add something like:
doc: $(OBJ)
doxygen with-correct-options
You will be able to generate the documentation using:
make doc
(doc being here the name of the target)
Now, if you add:
all: tree doc
#echo "Generating program and doc."
you will have the program and the documentation generated with simply invoking
make
In the end, there is an additional statment your Makefile could have use of: .PHONY. It's "A way to mark one of many targets as not directly producing files, and ensure their execution even if a file having the same name as the target exists". In other terms, it's to make sure doc, clean or all will always be executed even if files named doc, clean or all exist.
Its syntax is the following:
.PHONY: all clean doc
And is usually put at the end of the Makefile.