I am trying to parallelize an old Makefile.
In fact I need to make sure that some generator scripts are called not parallel before the compiling procedure starts.
The generators are always called before the compile. Is there any way to establish a somewhat critical section in a makefile?
The Makefile is messy, I'd prefer not to post it here.
You could put the rules for the generated files in a separate Makefile, and do something like this:
generated-sources :
$(MAKE) -f Makefile.Generators
However, with GNU Make, this will be parallel by default. You'll need to suppress it, and the manual isn't exactly clear how you would do that. Maybe some combination of the following would work, depending on which implementations of make you wish to support:
Pass the -j1 flag:
generated-sources :
$(MAKE) -j1 -f Makefile.Generators
Suppress MAKEFLAGS options:
generated-sources :
$(MAKE) -f Makefile.Generators MAKEFLAGS=
Suppress parallel execution with the following line in Makefile.Generators:
.NOTPARALLEL :
You can make a rule that runs all of the scripts, and then make sure all of your other rules depend on that rule.
all: ${SOURCES} scripts
scripts: last_script
first_script:
./script1.sh
second_script: first_script
./script2.sh
last_script: second_script
./script3.sh
source1.o: source1.c scripts
gcc source1.c -c -o source1.o
Related
I am currently learning C with the book "Learn C The Hard Way" and looked into reverse engineering as well. What I want to do is read through the compiler generated assembly for the C programs I have written. I know the option for gcc here is gcc -S -masm=intel file.c.
My question now is: can I automatically have gcc create the .s file and the .out file using the Makefile?
My previous Makefiles always looked like this:
CC=gcc
CLFAGS=-Wall -g
all: file
clean:
rm -f file
How can I extend my Makefile to make it work?
You could do something like the following:
CC=gcc
CFLAGS=-Wall -Werror -Wextra -O2 -g
SOURCE=file.c
all: binary assembly
binary: $(SOURCE)
$(CC) $(CFLAGS) $(SOURCE)
assembly: $(SOURCE)
$(CC) $(CFLAGS) -fverbose-asm -S -masm=intel $(SOURCE)
clean:
rm -f file.s a.out
And use it like this:
$ make # same as `make all`
$ make all # makes both executable (binary) and assembly
$ make binary # makes binary file only
$ make assembly # makes assembly only
It seems that you can execute other labels from other labels. However
how do they differ from just writing the desired filename like I did?
Remember that Makefiles consist of "rules" with the format:
target: dependencies
system command(s)
See more here.
The make program decides on what should be done based on how you invoke it and how you define your rules. For example, the first rule with target all depends on binary and assembly, so make checks those rules, and carries out the relevant steps (in this case executing gcc).
How does gcc differentiate between them?
The gcc program is a compiler, which is a completely separate program. It is merely invoked by make, just like rm is for the clean rule. It has no involvement in processing the Makefile itself, so it doesn't need to "differentiate" anything -- make does all the processing of the Makefile.
Also, the $() variables you use, do they only exist for gcc or could
you also define them globally in your operating system?
The variables have nothing to do with gcc. The make program parses the Makefile and performs the necessary substitutions before invoking the commands (e.g. gcc).
The variables could be used elsewhere in the Makefile too, for example with the rm command. They are not specific to any command in particular.
You can also make use of environment variables in a Makefile, as explained in this post.
The following terminal commands will compile and link my .c and .s files in linux
rm *.o
gcc -c printS.c -o printS.o
as lab8.S -o lab8.o
gcc lab8.o printS.o -o lab8test
I have never written a make file and I figure something like this deserves one. Any help is appreciated.
The same way you would make any make file. I am sure there are tons of examples out there and web pages.
ideally_the_result : dependencies
<tab>the commands
<tab>the commands
<tab>the commands
so
printS.o : printS.c
gcc -c printS.c -o printS.o
And make a rule for each of the commands you are using. I recommend you put the last one first so that it is the default if you dont have a command line option on make
also
clean :
rm -f *.o
rm -f lab8test
to cover cleanup (make clean)
That is the super simple way that just works. From there you can dig into all kinds of (make program specific, gnu make vs others) special characters that mean things to make more complicated rules.
This is my absolute first time ever making a makefile, and I'm really trying to understand the process.
I'm trying to create a very simple makefile for a C++ project whose structure is as follows:
root folder
makefile
readme
src folder
...source files all here...
include folder
...header files for external libraries here...
lib folder
...external lib files all here...
bin folder
...output directory for built executable...
obj folder
...object files all here...
I followed the tutorial here.
Here's my makefile:
IDIR=include .
CC=g++
CFLAGS=-I$(IDIR)
ODIR=bin/obj
LDIR=lib
LIBS=none
SRC=src
_DEPS=hello.h
DEPS=$(patsubst %,$(IDIR)/,%(_DEPS))
_OBJ=file1.o file2.o
OBJ=$(patsubst %,$(ODIR)/%,$(_OBJ))
$(ODIR)/%.o: $(SRC)/%.cpp $(DEPS)
$(CC) -c -o $# $< $(CFLAGS) # $(LIBS)
test_proj: $(OBJ)
$(CC) -o $# $^ $(CFLAGS)
.PHONY: clean
clean:
rm -f $(ODIR)/*.o *~ core $(INCDIR)/*~
When I run make on this, I get the following error:
g++ -o .o
g++: fatal error: no input files
compilation terminated.
<builtin>: recipe for target '.o' failed
mingw32-make.exe: *** [.o] Error 1
I'm using GNU Make 3.82.90 built for i686-pc-mingw32, if that matters at all.
Can anyone point out whatever ridiculous error I'm making?
IDIR=include .
is the first problem. Replace it by:
IDIR=include
With your code CFLAGS is expanded as:
-Iinclude .
It does not make sense, I'm afraid. The second problem is:
DEPS=$(patsubst %,$(IDIR)/,%(_DEPS))
which should probably be:
DEPS=$(patsubst %,$(IDIR)/%,$(_DEPS))
and would expand as:
DEPS=include/hello.h
if you fix the first problem, else as:
DEPS=include ./hello.h
which does not make sense neither. The cumulated effect of these two errors are strange recipes (I didn't try to expand them by hand) that probably trigger a make implicit rule with wrong parameters.
IDIR=include .
CC=g++
CFLAGS=-I$(IDIR)
This is wrong. First, for C++ code, use CXX not CC and CXXFLAGS not CFLAGS. Run make -p to understand the builtin rules of your make.
Then -I$(IDIR) does not "distribute" the -I, and IDIR is never used elsewhere. So I suggest to start your Makefile with:
CXX=g++
MY_CXX_LANG_FLAGS= -std=c++11
MY_CXX_WARN_FLAGS= -Wall -Wextra
MY_CXX_INCL_FLAGS= -I. -Iinclude
MY_CXX_MACRO_FLAGS= -DMYFOO=32
### replace with -O2 for a release build below
MY_CXX_OPTIM_FLAGS= -g
CXXFLAGS= $(MY_CXX_LANG_FLAGS) $(MY_CXX_WARN_FLAGS) \
$(MY_CXX_INCL_FLAGS) $(MY_CXX_MACRO_FLAGS)
I won't improve your Makefile, but I do suggest to upgrade to GNU make version 4 if possible (and compiling make 4.1 from its source code is worthwhile in 2015) for that purpose. If possible enable GUILE scripting in it.
If you are forced to use make 3.82 debug your Makefile using remake (with -x); if you can afford a make version 4 use its --trace option
BTW, you might consider using automatic dependencies, that is generating dependencies by passing -M or -MG (etc) flags of g++, see that.
At last, a simple project for a small program (less than a hundred thousands of source lines) might just put all (a few dozens of) its files in the current directory (then the Makefile could be simpler); your proposed directory structure might be arcane for a simple project (but could worth the pain if you have millions of C++ source lines of code). I've given several simple examples of Makefile, e.g. this & that. And GNU make source code itself has a less complex file tree that what you want.
BTW, I strongly disagree with the opinions of that answer (which I did upvote, since it is helpful). I don't feel that GNU make is senile, but I regret that, instead of using recent features available on recent versions (4.x) of make, many people prefer to use complex and arcane Makefile generators (like cmake) instead of coding a clever Makefile (for make version 4 specifically).
At last, you could use other builders, e.g. omake, icmake, ....
Is there a way of telling gcc to use the c99 standard when compiling c files as a default?
I want to avoid giving it the -std=c99 parameter all the time.
I assume I can do this by creating an alias in the .bashrc file, but seems to be rather inelegant.
You may call c99 instead of gcc (wrapper for gcc, if it's available on your system) or try to modify your gcc spec file. More information here: http://gcc.gnu.org/onlinedocs/gcc-4.7.0/gcc/Spec-Files.html
Here's an unexpected answer. Use a Makefile!
Pros:
Simply type make to build
All options are automatically handled in the build process.
While you're at it, go ahead and enable all warnings, as is good to do. See this.
Easy to scale up to multiple source files
Can handle multi-step builds involving different tools
Cons:
Another tool to learn, another thing to get wrong.
Consider this source:
#include <stdio.h>
int main() {
printf("Hello!\n");
int x = 4;
printf("%d\n", x);
return 0;
}
You could make a Makefile like this:
(Disclaimer, I don't actually know how to write them)
CC=gcc
CFLAGS=-Wall -pedantic -std=c99
LDFLAGS=
SOURCES=$(wildcard *.c)
OBJECTS=$(SOURCES:.cpp=.o)
EXECUTABLE=hello
.PHONY: clean
all: $(SOURCES) $(EXECUTABLE)
$(EXECUTABLE): $(OBJECTS)
$(CC) $(LDFLAGS) $(OBJECTS) -o $#
.cpp.o:
$(CC) $(CFLAGS) $< -o $#
clean:
rm -f *.o $(EXECUTABLE)
And it builds for me.
Likewise, if you remove the -std=c99, it shouldn't be valid C89 code, and indeed, typing make brings up the build error.
Custom compilation suggests you have at a working knowledge of compilers, standards, and basic flags / options. For that reason, I suggest you set shell variables in your .bashrc, .tcshrc, etc., and use them on the command line.
Since the choice of dialect can have an effect on linking: CC="gcc -std=c99", lets you invoke separate compilation commands using $CC -c -O2 ... foo.c, and is also picked up as the default for configure scripts, etc. Of course, you can always override a configure script with CC="gcc -std=c90" or CC="clang". The same applies to a preferred CFLAGS value, e.g.,
CFLAGS="-pipe -W -Wall -O2 -march=core2"
Allows for $CC $CFLAGS -c foo.c commands, and both environment variables are used by default with configure scripts, unless you explicitly override them. I think this is more useful than aliases. But perhaps I've just grown used to my own setup:)
Both of the proposed solutions are, in my opinion, almost what you want, but neither quite gets there.
Makefile solution
As seen here, by defining variables in your Makefile but not defining targets, you can use the make command like a customized pass-through to GCC. So if you create a Makefile in your "sandbox" directory (or wherever you're compiling outside of a real build system) and define the C*FLAGS vars, you'll essentially get what you want. An example Makefile:
CFLAGS=-Wall -std=c99
CXXFLAGS=-Wall -std=c++14
Now, make foo will turn foo.c into an executable called foo.
If you want to do this trick in multiple directories, put your makefile in a known location--say, ~/sandbox--and create the following alias (or something like it) in your .bashrc:
alias usestdmake="ln -s ~/sandbox/Makefile"
Then you can quickly compile a single file anywhere on your machine:
usestdmake
make foo
This has the added advantage of giving the output executable an appropriate name (foo in this case). It has the disadvantage of disabling tab-completion for your compile command (make fo<tab> does nothing, at least on my system).
Pure bashrc solution
The CC/CFLAGS variables mentioned in Brett Hale's answer are fairly standard, so it might be a good idea to define them in your bashrc. You can then use these variables inside of aliases.
In your .bashrc:
CFLAGS="-Wall -std=c99"
CC=gcc
# Use single-ticks to ensure that the variables are evaluated when the alias is evaluated.
alias mycc='$CC $CFLAGS'
On the command line:
cc foo.c # Compile with default CFLAGS
CFLAGS="$CFLAGS -O2" cc foo.c # Compile with modified CFLAGS
I have the following Makefile.am which is supposed to create foo.hdb and foo.cdb from foo.h (via the Python script):
TESTS = check_foo
check_PROGRAMS = check_foo
check_foo_SOURCES = check_foo.c $(top_builddir)/src/isti.h \
foo.cdb foo.h foo.hdb
check_foo_CFLAGS = #CHECK_CFLAGS# $(all_includes) -I../../clib/src/
check_foo_LDADD = $(top_builddir)/src/libcorm.la #CHECK_LIBS# -lsqlite3
%.hdb %.cdb: %.h
PYTHONPATH=$(top_builddir)/cgen/src python $(top_builddir)/cgen/src/isti/cgen/run.py $<
clean-local:
rm -f *.hdb *.cdb
However, although make foo.hdb and make foo.cdb work (call the Python code and generates the foo.hdb and foo.cdb files from foo.h), make clean check (or the two separately) does not (missing foo.hdb - no such file) - the pattern rule is not called to generate foo.hdb from foo.h.
In other words: the pattern rule is not being called for the files listed in check_foo_SOURCES.
How can I make this work? The rest of the autotools infrastructure is working fine. From looking at the Makefile I suspect the issue is with how autotools expands the check sources.
This is all on Linux with Gnu make. Here is the Makefile.
[Updated slightly to reflect the help from MadScientist].
Later update
The following Makefile (just make, not autotools) works fine, so the issue seems to be related to autotools and check support.
all: check_foo
CFLAGS=-I../../clib/src
LDFLAGS=-L../../clib/src/.libs
check_foo: check_foo.c foo.h corm_foo.h corm_foo.c
gcc $(CFLAGS) $(LDFLAGS) $^ -o $# -lcorm -lsqlite3
corm_%.h corm_%.c: %.h
PYTHONPATH=../../cgen/src python ../../cgen/src/isti/cgen/run.py $<
clean:
rm -f corm_*.h corm_*.c
rm -f *.o
(Note that I've switched from xxx.hdb to corm_xxx.h, etc, so that file extensions remain OK).
More Details
Since it seems to be related to the CHECK macros, this is configure.ac:
AC_INIT([corm], [0.1], [a.cooke#isti.com])
AC_CONFIG_MACRO_DIR([m4])
PKG_CHECK_MODULES([CHECK], [check >= 0.9.4])
AM_INIT_AUTOMAKE([-Wall foreign -Werror])
AC_PROG_CC_C99
AM_PROG_CC_C_O
LT_INIT
AC_CONFIG_HEADERS([config.h])
AC_CONFIG_FILES([Makefile clib/Makefile clib/src/Makefile clib/tests/Makefile clib/docs/Makefile cgen/Makefile cgen/docs/Makefile example/Makefile example/src/Makefile])
AC_CHECK_PROGS([DOXYGEN], [doxygen], AC_MSG_WARN([Doxygen not found - continuing without Doxygen support]))
AM_CONDITIONAL([HAVE_DOXYGEN], [test -n "$DOXYGEN"])
AM_COND_IF([HAVE_DOXYGEN], [AC_CONFIG_FILES([clib/docs/Doxyfile cgen/docs/Doxyfile])])
SOLUTION
OK, so summarizing the various things below, there were two important issues (once I had fixed file extensions - see the "plain" makefile and fceller's answer), either one of which was sufficient to make things work:
(The handling of) Header files is complicated. Because of auto-dependencies, programatically generated header files break things. The solution is to use BUILT_SOURCES
But (the handling of) .c files is not complicated. So putting the corm_foo.c in front of check_foo.c would trigger the generation of that file. Since that also generates corm_foo.h, everything works (because check_foo.c now can include corm_foo.h).
Also, fceller has some good general points about tidy makefiles and explains why the "plain" makefile works.
The line
%.cdb: %.hdb
does not do anything. Check the GNU make manual and you'll see that a pattern rule without a command line is used to DELETE a previously defined rule with that same pattern. Since there's no previous rule, this is essentially a no-op.
If you have a command that creates TWO output files with ONE invocation of a rule, then you need to put both patterns in the same rule, like this:
%.cdb %.hdb: %.h
PYTHONPATH=$(top_builddir)/cgen/src python $(top_builddir)/cgen/src/isti/cgen/run.py $<
This will tell GNU make that both targets are generated from one invocation of the rule. BE AWARE! This syntax only has this behavior for pattern rules. Explicit rules with multiple targets do something entirely different (confusingly enough).
As for the make clean behavior, I'm not sure. The makefile generated by automake is too complex for me to just read it; it would require some debugging (run it and trace what happens). However, I suspect that the rules there are not designed properly to allow a clean target and a build target to both be run in the same invocation of make. Run the two commands separately and see if that works better:
make clean
make check
You need to tell automake that foo.hdb is a source file that is to be constructed. Add the following to Makefile.am:
BUILT_SOURCES = foo.hdb
First of all: you do not need to include the "*.h" in *_SOURCES. The automake will generated code to generate the dependencies. From the manual: Header files listed in a _SOURCES definition will be included in the distribution but otherwise ignored
The change you made in the plain Makefile ("Note that I've switched from xxx.hdb to corm_xxx.h, etc, so that file extensions remain OK") is essential. The automake FILTERS the *_SOURCES list using the filename extension in order to see what to call (CC, CXX, F77).
The following Makefile.am will work:
TEST = check_foo
check_PROGRAMS = check_foo
check_foo_SOURCES = check_foo.c foo.db.c
check_foo_CFLAGS =
check_foo_LDADD =
%.db.c %.db.h: %.h
echo "int i = 1;" > foo.db.c
echo "int j;" > foo.db.h
clean-local:
rm -f *.db.h *.db.c
Let me try to add an indirect answer/discussion to the existing direct ones.
I recently moved away from make for the same kind of reasons you experienced: it is extremely powerful but sometimes a bit difficult to debug when things do not go as expected.
I recently discovered pydoit which is very promising as a debuggable replacement build tool for make. Since the notion of "pattern rules" was not present in it, I proposed an independent package to do the job: fprules.
This is how you would perform the same task that you mention in your post, with doit and fprules:
from fprules import file_pattern
# all: check_foo
DOIT_CONFIG = {'default_tasks': ['check_foo']}
CFLAGS = '-I../../clib/src'
LDFLAGS = '-L../../clib/src/.libs'
# check_foo: check_foo.c foo.h corm_foo.h corm_foo.c
# gcc $(CFLAGS) $(LDFLAGS) $^ -o $# -lcorm -lsqlite3
def task_check_foo():
"""
Compiles the `check_foo` executable
"""
src_files = ('check_foo.c', 'foo.h', 'corm_foo.h', 'corm_foo.c')
dst_file = 'check_foo'
return {
'file_dep': src_files,
'actions': ["gcc %s %s %s -o %s -lcorm -lsqlite3" % (CFLAGS, LDFLAGS, ' '.join(src_files), dst_file)],
'verbosity': 2,
'targets': [dst_file],
'clean': True
}
# corm_%.h corm_%.c: %.h
# PYTHONPATH=../../cgen/src python ../../cgen/src/isti/cgen/run.py $<
def task_gen_corm():
"""
Generates files `corm_%.h` and `corm_%.c`
for each header file `%.h`.
"""
for data in file_pattern('./*.h', dict(h_file='./corm_%.h', c_file='./corm_%.c')):
yield {
'name': data.name,
'file_dep': [data.src_path],
'actions': ["PYTHONPATH=../../cgen/src python ../../cgen/src/isti/cgen/run.py %s" % data.src_path],
'verbosity': 2,
'targets': [data.h_file, data.c_file],
'clean': True
}
# clean:
# rm -f corm_*.h corm_*.c
# rm -f *.o
# No need to create tasks for this:
# with 'clean': True, `doit clean` will clean all target files
Then simply run doit in the folder.
Do not hesitate to provide feedback on the projects pages if needed: for example multiline commands are not supported, you can vote for them if you too feel that they are missing: https://github.com/pydoit/doit/issues/314