I am trying to write a makefile for a small scale application I wrote in C under Linux. Currently all my source files .c are in the top level directory and all header files in
an include directory. Here is the makefile I used for this.
IDIR =include
CC=gcc
CFLAGS=-I$(IDIR)
ODIR=obj
_OBJ = main.o kernel.o user_app.o myargs.o ofp_msgs.o pkt_ip.o pkt_ether.o pkt_tcp.o pkt_udp.o pkt_icmp.o
OBJ = $(patsubst %,$(ODIR)/%,$(_OBJ))
#DEPS = ofp_msgs.h
$(ODIR)/%.o: %.c
$(CC) -c -o $# $< $(CFLAGS)
all: jam
jam: $(OBJ)
gcc -o $# $^ $(CFLAGS) -lpthread
.PHONY: clean
clean:
rm -f $(ODIR)/*.o *~ jam
It works fine but what I want is that for example I make a sub directory called "Packet" and all my packet parsing files i-e "pkt_ip.c, pkt_tcp.c etc" should be in that directory where as their header files should still be in the top level directory i-t "toplevel/include". I did a bit of search and the most common way was to use recursive make. Then I see a lots of pages complaining about recursive make. Can anyone please help me in this as how to do this right ?
Thanks
I recommend checking out the method described in Recursive Make Considered Harmful.
I have used it on several projects (small to medium-size), and find it simpler to use, and easier to wrap ones head around, than the recursive approach.
Basically, you have a Makefile in the root directory which includes a (partial) makefile from each of the subdirectories:
SRC := main.c
MODULES := Packet lib etc
-include $(patsubst %, %/module.mk, $(MODULES))
OBJ := $(patsubst %.c, %.o, $(filter %.c,$(SRC)))
# (...)
Packet/module.mk:
SRC += Packet/pkt_ip.c Packet/pkt_tcp.c
LIBS += -lsome_library
These module makefiles can of course also define their own module targets, or special build requirements.
Unlike recursive make, "make" will only be invoked once, which for most use cases will lead to a faster build.
However, for most smaller projects neither build time nor complexity will be a major concern, so use what feels most natural.
there are several ways to do this but you can certainly use VPATH:=Packet to tell make to look for source files inside the 'Packet' directory. see make manual
Related
Till now, I was using the following makefile that I have generated somehow for my school projects:
my makefile
But now I have a different situation: I am supposed to compile 4 programs for one project, while part of the code is supposed to be compiled as .so, for use for the 4 programs.
like described here:
1 - all the parts that are supposed to be compiled together as one .so file, using for example:
gcc -shared -fPIC src/file1.c src/file2.c src/file3.c -o libutils.so
3,4,5 should be compiled and linked together with this .so file, using for example:
gcc src/file4.c -L'pwd' lutils -o file4.out
the same way for all the 3 projects, and one more simple compilation of project 2.
I wandered across the net, google, your site, etc.
tried to find a solution for this situation,
without any luck.
already seen solutions like this one:
solution example
where you supply makefile with the details of the entire project structure.
I thought about dividing all the files into 4 folders, below the main folder, and creating a loop inside makefile that will compile each program in each cycle, with "if" statements to make a different compilation, according to the index. but I had no luck, it seems very complicated (maybe someone can show me a solution like that one...).
I am wondering if there is a way of making this whole compilation process generic and automatic like the current file (maybe little less),
if there is a way, I would like to study and discover it.
thank you in advance!!!
Arie
Since you have a nicely drawn tree of dependencies, you "just" need to translate this into a Makefile.
You might like to start with this:
.PHONY: all
all: reloader.exe block_finder.exe formatter.exe printdb.exe
MODULES = reloader block_finder formatter printdb linked_list bitcoin file_handler
SRCS = $(MODULES:%=%.c)
reloader.exe block_finder.exe formatter.exe printdb.exe: libbitcoin_manager.so
reloader.exe: reloader.o
block_finder.exe: block_finder.o
formatter.exe: formatter.o
printdb.exe: printdb.o
libbitcoin_manager.so: linked_list.o bitcoin.o file_handler.o
gcc -shared -fPIC $^ -o $#
%.exe: %.o
gcc $< -L. -lbitcoin_manager -o $#
%.o: %.c
gcc -c $< -o $#
%.d: %.c
gcc -MM -MT $# -MT $*.o -MF $# $<
include $(SRCS:%.c=%.d)
Because you don't have a loop in the diagram, you don't need a loop in the Makefile. Instead you put all dependent files on the left of a colon and the file they depend on on the right.
You might like to collect more "objects" in variables, for example the programs to build, the modules in the library, and so on.
I have also used a common pattern to generate the dependencies from the header files. The way shown is just one way to do it. It uses files with a ".d" extension, for "dependency." GCC has options to build these files, it scans the source and collects all included headers even if "stacked."
For example, "bitcoin.d" looks like this:
bitcoin.d bitcoin.o: bitcoin.c bitcoin.h linked_list.h definitions.h \
file_handler.h
The re-generate the dependency file on changes in the sources it is also a target, not only the object file.
EDIT:
First, using directories makes Makefiles more difficult. I don't like such structures not only for that reason, but also because they separate header files and implementation files that clearly belong to each other.
Anyway, here is an enhanced Makefile:
.PHONY: all
SRCDIR = src
INCDIR = include
BLDDIR = build
APPS = reloader block_finder formatter printdb
MODULES = reloader block_finder formatter printdb linked_list bitcoin file_handler
LIBNAME = bitcoin_manager
LIBMODULES = linked_list bitcoin file_handler
VPATH = $(SRCDIR)
SRCS = $(MODULES:%=%.c)
LIB = $(LIBNAME:%=lib%.so)
#win LIB = $(LIBNAME:%=%.lib)
EXES = $(APPS:%=%.exe)
all: $(BLDDIR) $(EXES)
$(BLDDIR):
mkdir $#
$(LIB): $(LIBMODULES:%=$(BLDDIR)/%.o)
gcc -shared -fPIC $^ -o $#
$(EXES): $(LIB)
$(EXES): %.exe: $(BLDDIR)/%.o
gcc $< -L. -l$(LIBNAME) -o $#
$(BLDDIR)/%.o: %.c
gcc -I$(INCDIR) -c $< -o $#
$(SRCDIR)/%.d: %.c
gcc -I$(INCDIR) -MM -MT $# -MT $(BLDDIR)/$*.o -MF $# $<
include $(SRCS:%.c=$(SRCDIR)/%.d)
It uses a lot more variables to simplify renaming and managing a growing library and application.
One important issue is the use of VPATH. This makes make search for sources in the list of paths assigned to it. Make sure you understand it thoroughly, search for articles and documentation. It is easy to use it wrong.
The pattern $(EXES): %.exe: $(BLDDIR)/%.o is a nice one. It consists of three parts, first a list of targets, second a generic pattern with a single target and its source. Here is means that for all executables each of them is built from its object file.
Now to your questions:
Is answered by the new proposal. I didn't add the directory but use VPATH.
Make stopped not because the exe-from-o pattern was wrong, but because it didn't find a way to build the object file needed. This is solved by the new proposal, too. To find out what happens if you delete these 4 recipes in the old proposal: you can experiment, so do it!
The dot is, like user3629249 tried to say, the present working directory. You had it in your Makefile with 'pwd' and I replaced it. This is not special to make, it is common in all major operating systems, including Windows. You might know .. which designates the parent directory.
When make starts it reads the Makefile or any given file. If this file contains include directives the files listed are checked if they need to be rebuild. make does this even if you call it with -n! After (re-)building all files to be included they are included finally. Now make has all recipes and continues with its "normal" work.
I am very new at make. Thus far I've managed to create the following using some of GNU manual and tutorials found online. I'd like for make to place all of the created object files into the directory 'obj.' I've been able to successfully create this directory, but I cannot figure out how to place the files in it. Any suggestions or tips are appreciated. Also, on a general note, is there a good source for learning how to work with make besides the GNU documentation?
# specify compiler
CC := gcc
# set compiler flags
CFLAGS := -M -Igen/display -Igen/logic -Iman -Ilib/include -pipe -march=native -ftime-report
# set linker flags
LDFLAGS := -lglut32 -loglx -lopengl32 -Llib
# specify separate directory for objects
OBJDIR := obj
# include all sources
SOURCES := $(wildcard gen/display/*.c gen/logic/*.c man/*.c)
# create objects from the source files
OBJECTS := $(patsubst %.c,%.o,$(SOURCES))
# specify the name and the output directory of executable
EXECUTABLE := win32/demo
# all isn't a real file
all: $(EXECUTABLE)
# compile
%.o: %.c
#mkdir -p $(#D)
$(CC) $(CFLAGS) -c $< -o $(OBJDIR)/$#
# link
$(EXECUTABLE): $(OBJECTS)
$(CC) $^ $(LDFLAGS) -o $#
# clean objects
clean:
#$(RM) -rf $(OBJDIR)
.PHONY: all clean
Any time you see a rule where the output generated does not go to the file $#, you know it's not right. Make will set the $# automatic variable to the file name that it expects to be created and if the recipe does something different, the makefile will not work.
Your rule sends the file to $(OBJDIR)/$#, not $#, so it's not right.
So, you need to write your pattern rule like this:
%.o: %.c
#mkdir -p $(#D)
$(CC) $(CFLAGS) -c $< -o $#
If that doesn't work you'll need to provide more information such as an example of the compile line make invokes, what errors you see, etc.
I've been able to successfully create this directory, but I cannot
figure out how to place the files in it.
There are two parts writing to doing that explicitly.
First, and most fundamental, is that if you want make to create a file, you have to give it a rule for doing so. You do have a pattern rule that could, in principle, have that effect ...
%.o: %.c
# ...
... but in practice, that rule cannot ever be matched to files in the obj/ directory because your sources are not in that directory. This might be more effective:
$(OBJDIR)/%.o: %.c
#mkdir -p $(#D)
$(CC) $(CFLAGS) -c $< -o $#
Note in particular how now the target of the rule matches the artifact actually produced by that rule.
Second, you must have a requirement to build the target of the rule, usually by having it be a dependency of some other rule. Observe that your variable defining the object files contributing to $(EXECUTABLE) does not rely on objects from the obj/ directory. It is generated by this pattern substitution ...
OBJECTS := $(patsubst %.c,%.o,$(SOURCES))
... which generates object names with the same path as the corresponding sources. You probably want something more like this:
OBJECTS := $(patsubst %.c,$(OBJDIR)/%.o,$(SOURCES))
You will note how that also corresponds to the change presented in the previous point.
But that's a lot of work for little gain. You would not have to modify your clean target very much to do without it there. You could write your file in a somewhat simpler and more conventional form and still get output into a separate directory by leveraging the VPATH feature of GNU (and some other) make.
I'd like to write a simple Makefile which creates all *.o, *.so, and binary files in a build directory, possibly build/.
Is there a straightforward way of doing this in a Makefile? I'm on Linux, Ubuntu 14.04.
The linked question puts all *.o artifacts in a build directory, but not the executables themselves. To be clear, I'd like to hit make and have all items compiled and stored in build/.
Assuming you use GNU make, something like:
O ?= build
OBJS := $(patsubst %.c,$(O)/%.o,$(wildcard *.c))
SOS := $(O)/<you know better than me>
EXECS := $(O)/<you know better than me>
all: $(OBJS) $(SOS) $(EXECS)
$(OBJS): $(O)/%.o: %.c
mkdir -p $(O); \
$(CC) $(CFLAGS) -c $< -o $#
$(SOS): ...
$(EXECS): ...
should be close to what you want. As I do not know which *.so and executables you want to build and how, I just indicate a possibility for the rule that builds the *.o. It should be easy to adapt to the other targets.
If you type make, the build sub-directory will be created if it does not exist. If you type make O=foo, the foo sub-directory will be created (if it does not exist) and be used instead of build.
$(OBJS): $(O)/%.o: %.c is a static pattern rule. In case it is not clear enough, the GNU make documentation will tell you everything about them.
I have created a Makefile for unit tests which uses GCC with arguments to create profiling files (gcno) during compiling. Here's a similified part of it where compiling and linking takes place:
UTEXE = $(UTOBJSDIR)\$(UTUNIT).exe
UTOBJS = $(UTUUTSRC:.c=.o) $(UTUTSRC:.c=.o) $(UTCSRC:.c=.o)
UTOBJSFULL = $(addprefix $(UTOBJSDIR)\,$(UTOBJS))
UTOBJSGCNO = $(addprefix $(UTOBJSDIR)\,$(UTOBJS:.o=.gcno))
$(UTOBJS): %.o: %.c $(UTMAKEDEP)
$(call report,Compiling $(*F).c)
$(MKDEP) $(MKDFLAGS) -o.o -f$(UTOBJSDIR)\$(*F).dep $(subst /,\,$<)
$(CC) -c $(CFLAGS) $(subst /,\,$<) -o $(UTOBJSDIR)/$#
$(UTOBJSGCNO): $(UTOBJS) $(UTMAKEDEP)
utbuild: $(UTEXE) $(UTOBJSGCNO) $(UTOBJS) $(UTMAKEDEP)
$(UTEXE): $(UTOBJSGCNO) $(UTOBJS) $(UTMAKEDEP)
$(call report,Linking to $(UTUNIT).exe)
$(LINK) $(UTOBJSFULL) $(LNKFLAGS) -o $(UTEXE)
It compiles all the object and profile files and links together a binary. However when i delete some profile file (gcno) and call "utbuild" again it won't re-compile to restore the .gcno file. It tries to do linking again because gcno is a prequisite to it, but it wont do the compiling.
I don't know how to name this case so couldn't find solution from internet. Basically one recipe creates two files and i don't know how to write the rule that re-run's recipe even when only one file needs to re-created.
I would appreciate some links or hints.
thanks for all the comments. I've tried no-op ";" and ":=" with same outcome.
I think i need to take one step back and explain why i asked this question. It's not just about deleting or not-deleting gcno files manually, it's about general understanding how to write such a Makefile which restores any missing or out-of-date file. My Makefile has similar cases in few places and it's using parallel build so when some file goes missing it gives lot of weird errors. Usually it's solved by "clean" and "all", but i'd like the Makefile to be perfect and handle the missing file issues nicely.
As the example above is not so clear without all the rest of the Makefile then i made a new simple test.
hello.c
#include <stdio.h>
int main()
{
printf("Hello world\n");
}
Makefile
CCDIR = C:\tools\MinGW
CCBINDIR = $(CCDIR)\bin
CCINCDIR = $(CCDIR)\include;$(CCDIR)\lib\gcc\mingw32\4.8.1\include
CCLIBDIR = $(CCDIR)\lib;$(CCDIR)\lib\gcc\mingw32\4.8.1
# Overcome "missing dll file" messages on Windows
CC = set PATH=%PATH%;$(CCBINDIR)& $(CCBINDIR)\gcc.exe
LINK = set PATH=%PATH%;$(CCBINDIR)& $(CCBINDIR)\gcc.exe
# Compile and link for code coverage
CFLAGS = -fprofile-arcs -ftest-coverage -g3 -O0 $(addprefix -I,$(CCINCDIR))
LNKFLAGS = -fprofile-arcs -ftest-coverage -static -static-libgcc $(addprefix -L,$(CCLIBDIR))
OBJECTS = hello.o
EXE = hello.exe
$(OBJECTS): %.o: %.c
$(CC) -c $(CFLAGS) $(subst /,\,$<) -o $#
$(EXE): $(OBJECTS)
$(LINK) $(OBJECTS) $(LNKFLAGS) -o $(EXE)
build: $(EXE)
"make build" creates following files:
hello.o
hello.gcno
hello.exe
Now if i delete the "hello.gcno" and run build again it tells me:
mingw32-make: Nothing to be done for 'build'.
The goal is to update Makefile so that the make re-creates the "hello.gcno". It would probably re-create "hello.o" and "hello.exe" also during that process but that's not a problem.
Edit:
Just to be clear: in real Makefile i really-really need the .gcno files. It's not just an additional information or something which to avoid or do optionally. The Makefile builds the unit test executables, runs them and executes gcov to generate code coverage information and gcovr creates a report of all the .gcov files. If .gcno file is missing it won't work. Also - as it's parallel build then dependencies shall be absolutely correct to avoid some process starting earlier and it's tricky because coverage report has dependencies coming from two "branches" - .gcno files from compile stage and .gcda files from execute stage. So that's why i need it to be correct.
your only option here is this :
(if you can change the rule)
$(EXE): $(OBJECTS)
$(LINK) $(OBJECTS) $(LNKFLAGS) -o $(EXE)
to this:
%.exe %.gnco: $(OBJECTS)
$(LINK) $(OBJECTS) $(LNKFLAGS) -o $(EXE)
$(GENERATE_GNCO) $<
Here is a very simple solution of one thing dependig on two other things
compile:./src/main.c ./src/error.c
gcc ./src/error.c ./src/main.c -o ./exe/calc
run : ./exe/calc
./exe/calc
The correct answer in my opinion, is, don't delete any .gcno files by themselves. If you have to "clean", use make clean, but don't just go about deleting files.
The "build" is a state machine, with all the files constituting a "state". Don't corrupt the state!
Some people say, one should be able to delete arbitrary files and the build should recover. My answer is, what about if you corrupt some .o file by hand, say, add some 0's and 1's, making it unusable (thank you user3629249 for pointing that needs to be clarified, that I am talking about corruption, not intentional editing). Should the build also recover from that? Obviously no - no build system in the world will recover if you touch the .o file this way. Then why allow deleting a file, but not allow modifying it?? Where do you draw the line?
Simply put, any corruption should not be allowed. Use make clean only, or better yet, write your Makefile properly, so you never need to clean period.
The whole Makefile has a number of problems, here is how it should look like (I am assuming this is on Windows/DOS):
.SUFFIXES:
UTEXE := $(UTOBJSDIR)\$(UTUNIT).exe
UTOBJSFULL := $(addprefix $(UTOBJSDIR)\,$(subst /,\, $(UTUUTSRC:.c=.o) $(UTUTSRC:.c=.o) $(UTCSRC:.c=.o)))
UTOBJSGCNO := $(UTOBJSFULL:.o=.gcno)
.PHONY: utbuild all
all: utbuild
utbuild: $(UTEXE) $(UTOBJSGCNO) $(UTMAKEDEP)
$(UTOBJSGCNO): %.gcno: %.o $(UTMAKEDEP) ;
.SECONDARY: %\.
%\.: Makefile
mkdir $*
.SECONDEXPANSION:
$(UTOBJSFULL): $(UTOBJSDIR)\%.o: %.c $(UTMAKEDEP) | $$(#D)\.
$(call report,Compiling $<)
$(MKDEP) $(MKDFLAGS) -o.o -f$(UTOBJSDIR)\$(*F).dep $<
$(CC) -c $(CFLAGS) $< -o $#
$(UTEXE): $(UTOBJSFULL) $(UTMAKEDEP) | $$(#D)\.
$(call report,Linking to $(#F))
$(LINK) $(UTOBJSFULL) $(LNKFLAGS) -o $#
This makefile does not behave as I expect. I want it to build .o files for each .c file in the current directory and subdirectories, and put them in a static library. However, it stops applying my $(INCS) after the first or second file. When it tries to build the second .o file, I don't see the -I paths in the build line and it complains about not finding a header file therein. Names have been genericized to simplify things. I'm using cygwin on Windows XP. I'm using an ARM cross compiler that is not under the cygwin tree. I based this makefile off an answer here. There are only about two dozen .c files so the overhead of creating the dependency files this way isn't a big deal.
# Project specific options
CC = my-cross-gcc
INCS := -I. -Iinc
INCS += -Imy/inc/path
CFLAGS := -Wall -fPIC -static -cross-compiler-specific-options
OUT := bin/libmylib.a
MKDIR:=mkdir -p
### Generic C makefile items below:
# Add .d to Make's recognized suffixes.
SUFFIXES += .d
NODEPS:=clean
#Find all the C files in this directory, recursively
SOURCES:=$(shell find . -name "*.c")
#These are the dependency files
DEPFILES:=$(patsubst %.c,%.d,$(SOURCES))
OBJS:= $(patsubst %.c,%.o,$(SOURCES))
#Don't create dependencies when we're cleaning, for instance
ifeq (0, $(words $(findstring $(MAKECMDGOALS), $(NODEPS))))
-include $(DEPFILES)
endif
#This is the rule for creating the dependency files
%.d: %.c
$(CC) $(INCS) $(CFLAGS) -MM -MT '$(patsubst %.c, %.o,$(patsubst %.c,%.o,$<))' $< > $#
#This rule does the compilation
%.o: %.c %.d %.h
$(CC) $(INCS) $(CFLAGS) -o $# -c $<
# Now create a static library
all: $(OBJS)
#$(MKDIR) bin
ar rcsvq $(OUT) $(OBJS)
clean:
rm -rf $(OBJS) $(OUT) $(DEPFILES)
Why does this makefile not apply $(INCS) when building subsequent .o files? How do I fix it? Output resembles this:
$ make all
my-cross-gcc -I. -Iinc -Imy/inc/path -<compiler options> -o firstfile.o -c firstfile.c
my-cross-gcc -I. -Iinc -Imy/inc/path -<compiler options> -o secondfile.o -c secondfile.c
my-cross-gcc -<compiler flags> -o thirdfile.o -c thirdfile.c
thirdfile.c:23:18: fatal error: myinc.h: No such file or directory
compilation terminated.
When I go to the command line and type in the gcc line to build thirdfile.o and use the -I paths, the object file is successfully built.
There are two different mechanisms for handling header files at work here:
When the compiler is trying to build foo.o from foo.c, and in foo.c it encounters #include "foo.h", it goes looking for foo.h. The -I flags tell it where to look. If it is invoked without the flags it needs to find foo.h, it will complain and die.
When Make is trying to build foo.o, and considering which rule to use, it looks at the prerequisites. The prerequisites for your rule are foo.c foo.d foo.h, so it goes looking for those prerequisites. How is it to know where foo.h is? Note that the compiler flag inside one of its commands is of no use-- it won't make any deductions about that. If it can't find (and doesn't know how to make) a prerequisite, it will reject that rule and look for another one, such as the implicit %.o rule which knows nothing about your $(INCS) variable, and that leads you to the problem described above.
If this is the problem (and you can check by looking at the locations of the headers and doing some experiments) you have a couple of options:
A) You can use the implicit rule, and it's variables. Just add INCS to CFLAGS and you'll probably get the results you want. This tells the compiler what to do, but it still leaves Make in the dark about the dependencies, so you'll probably have to double-check that your dependency handling is correct.
B) You can tell Make where to find the header files:
vpath %.h inc my/inc/path
(You may notice that this is redundant with your INCS variable, and redundancy is bad-- you can eliminate this redundancy, but I urge you to get it working first.)
I'm going to guess that you have files named firstfile.h, secondfile.h, but no file named thirdfile.h?
I would then suppose that make cannot use the rule you gave it because and can't find or build the .h file. So it decides to use the default implicit rule instead.
All I can imagine is that for "thirdfile" your depfile is somehow out-of-date or corrupt. Perhaps it is bad enough that it's confusing make into calling some other default target.