Complex Makefile with Python and Shared Libraries - c

I have next to no experience in Makefiles, and I'm trying to learn - and I understand most of it. But, I have a directory structure like this:
/src/
/mysql/
mp_mysql.h
mp_mysql.c
/sqlite/
mp_sqlite.h
mp_sqlite.c
/lib/
mysql.so
sqlite.so
And I'm trying to write a makefile that compiles the files in /src/mysql into /lib/mysql.so and do the same for sqlite. I could probably get it if someone helped me along! Thanks!
P.S. The project is a Python project, and I'm writing a special adapter in C for MySQL. All of the resources I've found online I can't get to work and/or work to make one big executable.

First, write a Makefile for each directory and define a DIR_BUILD variable, which can be overridden when calling make.
Then, write a project Makefile which runs each of the child Makefiles.
For example,
DIR_BUILD = '.'
mysql.so: mp_mysql.c mp_mysql.h
gcc $< -o $(DIR_BUILD)/$#
and, for the parent Makefile:
SUBDIRS = `find . -mindepth 1 -maxdepth 1 -type d`
all:
for d in $(SUBDIRS); \
do \
$(MAKE) --directory=$$d DIR_BUILD=../lib; \
done
No guarentees, let me know how it goes!

Related

How to write a Makefile to compile a simple C program

Compile the following program
#include <stdio.h>
int main(void)
{
printf ("Hello from your first program!\n");
return 0;
}
a)-by using file of type Makefile
b)-the executable will be named Hello
"Please help to do an exercise. I know how to do it in CodeBlocks, but I don't know what Makefile is and how to write it in Linux. I compiled it using command "gcc filename.c" and subsequently "./a.out" but I still don't understand what the Makefile is. Is it a sort of shell script, an instruction? How would a Makefile for this task exactly look? Thanks in advance :) "
This is your simple make file for hello program.
CC = gcc
CFLAGS = -g
RM = rm -f
default: all
all: Hello
Hello: Hello.c
$(CC) $(CFLAGS) -o Hello Hello.c
clean veryclean:
$(RM) Hello
Suppose you have two makefiles in one directory named makefile.m1 and makefile.m2 and if you want build both make file then please use following commands
make -f makefile.m1
make -f makefile.m2
or use single Makefile that contains:
m1:
make -f makefile.m1
m2:
make -f makefile.m2
and use make m1 or make m2
Now lets clear your doubt about name of make file must not require Makefile
You can name makefile whatever you want. suppose i would like to give name myfirstmakefile.mk. To use it later you need to tell make what makefile you want. Use -f option for this:
make -f myfirstmakefile.mk
And again extantion .mk is also not manadatory you can use whatever you want but never forgot to use -f option.
so may this help make sense to you.
A makefile is a recipe for the make utility how to create some file (called a target) from some other files (called dependencies) using a set of commands run by the shell. A makefile typically looks like this:
target: dependency [...]
command1
command2
Try running man make for details.
Now for your task, really there is no need for a Makefile, since make has built-in rules that know how to compile a simple program. All you need to do is place your C source in a file named after the executable name (Hello) and with a .c extension, i.e. Hello.c.
Then a simple
$ make Hello
cc Hello.c -o Hello
does everything. If you want to use gcc instead of cc, you can run
$ rm Hello
$ make CC=gcc Hello
gcc Hello.c -o Hello
If you tell your instructor/teacher/prof that an empty makefile is all you need since you know the built-in rules do the right thing, you'll get some extra credit and maybe your instructor has learnt something new :-) If you are asked for a reference, you could quote the relevant parts of the make manual, or, do it like a pro, quote from the POSIX Standard for the make utility, section Default Rules.
before going for makefile you have to know what's it and why we need it
What is Makefile?
Makefile is a script written in a certain prescribed syntax which helps to build the target output (normally, one or more executables) from source files by compilation and linking. In simple words, makefile will compile your source code in simple & fast way.
Why we need Makefile?
=> Large projects can contain multiple source files which are dependent in one another or arranged in hierarchical manner for example, in order to compile file A, you have to first compile B; in order to compile B, you have to first compile C; and so on.
=> Make is a solution to these problems. It can be used to compile whole project in well arranged manner and generate your target according to your make rule(which we will discuss later) by entering single command that is make.
=> An important feature is that when a project is recompiled after a few changes, it will recompile only those files which are changed, and any other files that are dependent on it. This saves a lot of time.
=> For a large project, when a few changes are made to the source, manually recompiling the entire project each time is tedious, error-prone and time-consuming.
Here is nice link for it :How to write first makefile
A makefile is a recipe for computers with instructions how to perform certain tasks and with dependencies between those tasks.
In the simple form, it looks like so:
a.out: filename.c
gcc filename.c
Read: "To build a.out from filename.c, run the command gcc filename.c. If a.out is newer than filename.c, then don't do anything"
Note: The first character in the gcc line must be a tab.

Automake, GNU make, check, and (ignored) pattern rules

I have the following Makefile.am which is supposed to create foo.hdb and foo.cdb from foo.h (via the Python script):
TESTS = check_foo
check_PROGRAMS = check_foo
check_foo_SOURCES = check_foo.c $(top_builddir)/src/isti.h \
foo.cdb foo.h foo.hdb
check_foo_CFLAGS = #CHECK_CFLAGS# $(all_includes) -I../../clib/src/
check_foo_LDADD = $(top_builddir)/src/libcorm.la #CHECK_LIBS# -lsqlite3
%.hdb %.cdb: %.h
PYTHONPATH=$(top_builddir)/cgen/src python $(top_builddir)/cgen/src/isti/cgen/run.py $<
clean-local:
rm -f *.hdb *.cdb
However, although make foo.hdb and make foo.cdb work (call the Python code and generates the foo.hdb and foo.cdb files from foo.h), make clean check (or the two separately) does not (missing foo.hdb - no such file) - the pattern rule is not called to generate foo.hdb from foo.h.
In other words: the pattern rule is not being called for the files listed in check_foo_SOURCES.
How can I make this work? The rest of the autotools infrastructure is working fine. From looking at the Makefile I suspect the issue is with how autotools expands the check sources.
This is all on Linux with Gnu make. Here is the Makefile.
[Updated slightly to reflect the help from MadScientist].
Later update
The following Makefile (just make, not autotools) works fine, so the issue seems to be related to autotools and check support.
all: check_foo
CFLAGS=-I../../clib/src
LDFLAGS=-L../../clib/src/.libs
check_foo: check_foo.c foo.h corm_foo.h corm_foo.c
gcc $(CFLAGS) $(LDFLAGS) $^ -o $# -lcorm -lsqlite3
corm_%.h corm_%.c: %.h
PYTHONPATH=../../cgen/src python ../../cgen/src/isti/cgen/run.py $<
clean:
rm -f corm_*.h corm_*.c
rm -f *.o
(Note that I've switched from xxx.hdb to corm_xxx.h, etc, so that file extensions remain OK).
More Details
Since it seems to be related to the CHECK macros, this is configure.ac:
AC_INIT([corm], [0.1], [a.cooke#isti.com])
AC_CONFIG_MACRO_DIR([m4])
PKG_CHECK_MODULES([CHECK], [check >= 0.9.4])
AM_INIT_AUTOMAKE([-Wall foreign -Werror])
AC_PROG_CC_C99
AM_PROG_CC_C_O
LT_INIT
AC_CONFIG_HEADERS([config.h])
AC_CONFIG_FILES([Makefile clib/Makefile clib/src/Makefile clib/tests/Makefile clib/docs/Makefile cgen/Makefile cgen/docs/Makefile example/Makefile example/src/Makefile])
AC_CHECK_PROGS([DOXYGEN], [doxygen], AC_MSG_WARN([Doxygen not found - continuing without Doxygen support]))
AM_CONDITIONAL([HAVE_DOXYGEN], [test -n "$DOXYGEN"])
AM_COND_IF([HAVE_DOXYGEN], [AC_CONFIG_FILES([clib/docs/Doxyfile cgen/docs/Doxyfile])])
SOLUTION
OK, so summarizing the various things below, there were two important issues (once I had fixed file extensions - see the "plain" makefile and fceller's answer), either one of which was sufficient to make things work:
(The handling of) Header files is complicated. Because of auto-dependencies, programatically generated header files break things. The solution is to use BUILT_SOURCES
But (the handling of) .c files is not complicated. So putting the corm_foo.c in front of check_foo.c would trigger the generation of that file. Since that also generates corm_foo.h, everything works (because check_foo.c now can include corm_foo.h).
Also, fceller has some good general points about tidy makefiles and explains why the "plain" makefile works.
The line
%.cdb: %.hdb
does not do anything. Check the GNU make manual and you'll see that a pattern rule without a command line is used to DELETE a previously defined rule with that same pattern. Since there's no previous rule, this is essentially a no-op.
If you have a command that creates TWO output files with ONE invocation of a rule, then you need to put both patterns in the same rule, like this:
%.cdb %.hdb: %.h
PYTHONPATH=$(top_builddir)/cgen/src python $(top_builddir)/cgen/src/isti/cgen/run.py $<
This will tell GNU make that both targets are generated from one invocation of the rule. BE AWARE! This syntax only has this behavior for pattern rules. Explicit rules with multiple targets do something entirely different (confusingly enough).
As for the make clean behavior, I'm not sure. The makefile generated by automake is too complex for me to just read it; it would require some debugging (run it and trace what happens). However, I suspect that the rules there are not designed properly to allow a clean target and a build target to both be run in the same invocation of make. Run the two commands separately and see if that works better:
make clean
make check
You need to tell automake that foo.hdb is a source file that is to be constructed. Add the following to Makefile.am:
BUILT_SOURCES = foo.hdb
First of all: you do not need to include the "*.h" in *_SOURCES. The automake will generated code to generate the dependencies. From the manual: Header files listed in a _SOURCES definition will be included in the distribution but otherwise ignored
The change you made in the plain Makefile ("Note that I've switched from xxx.hdb to corm_xxx.h, etc, so that file extensions remain OK") is essential. The automake FILTERS the *_SOURCES list using the filename extension in order to see what to call (CC, CXX, F77).
The following Makefile.am will work:
TEST = check_foo
check_PROGRAMS = check_foo
check_foo_SOURCES = check_foo.c foo.db.c
check_foo_CFLAGS =
check_foo_LDADD =
%.db.c %.db.h: %.h
echo "int i = 1;" > foo.db.c
echo "int j;" > foo.db.h
clean-local:
rm -f *.db.h *.db.c
Let me try to add an indirect answer/discussion to the existing direct ones.
I recently moved away from make for the same kind of reasons you experienced: it is extremely powerful but sometimes a bit difficult to debug when things do not go as expected.
I recently discovered pydoit which is very promising as a debuggable replacement build tool for make. Since the notion of "pattern rules" was not present in it, I proposed an independent package to do the job: fprules.
This is how you would perform the same task that you mention in your post, with doit and fprules:
from fprules import file_pattern
# all: check_foo
DOIT_CONFIG = {'default_tasks': ['check_foo']}
CFLAGS = '-I../../clib/src'
LDFLAGS = '-L../../clib/src/.libs'
# check_foo: check_foo.c foo.h corm_foo.h corm_foo.c
# gcc $(CFLAGS) $(LDFLAGS) $^ -o $# -lcorm -lsqlite3
def task_check_foo():
"""
Compiles the `check_foo` executable
"""
src_files = ('check_foo.c', 'foo.h', 'corm_foo.h', 'corm_foo.c')
dst_file = 'check_foo'
return {
'file_dep': src_files,
'actions': ["gcc %s %s %s -o %s -lcorm -lsqlite3" % (CFLAGS, LDFLAGS, ' '.join(src_files), dst_file)],
'verbosity': 2,
'targets': [dst_file],
'clean': True
}
# corm_%.h corm_%.c: %.h
# PYTHONPATH=../../cgen/src python ../../cgen/src/isti/cgen/run.py $<
def task_gen_corm():
"""
Generates files `corm_%.h` and `corm_%.c`
for each header file `%.h`.
"""
for data in file_pattern('./*.h', dict(h_file='./corm_%.h', c_file='./corm_%.c')):
yield {
'name': data.name,
'file_dep': [data.src_path],
'actions': ["PYTHONPATH=../../cgen/src python ../../cgen/src/isti/cgen/run.py %s" % data.src_path],
'verbosity': 2,
'targets': [data.h_file, data.c_file],
'clean': True
}
# clean:
# rm -f corm_*.h corm_*.c
# rm -f *.o
# No need to create tasks for this:
# with 'clean': True, `doit clean` will clean all target files
Then simply run doit in the folder.
Do not hesitate to provide feedback on the projects pages if needed: for example multiline commands are not supported, you can vote for them if you too feel that they are missing: https://github.com/pydoit/doit/issues/314

compile C program in multiple folders

I have a bunch of directories with C programs. I need to compile them one by one and use the result in the main Program.
So my Main program traverse in the Directory structure [I am not sure what the structure is, It may change over time] and compiles one C program at a time, use that result in some computation.
So If I write the main program in C and use nftw to traverse.
OR
Write a Shell main program.
Which appraoch is better?
I guess you use a compatible Unix/Linux/Cygwin system....
Therefore, I would advice to use a shell solution because it is more suitable for directory processing.
Makefile for each program
As #Dogbert said make can be used to build several programs. Build can be performed in parallel (using the option -j). Moreover make can also take care about the dependencies.
I do not know if you are used with Makefile syntax. Therefore I give a quick example about a program requiring three C files and one header file:
program: file1.o file2.o file3.o
gcc -o -o $# $^
%.o: %.c header.h
gcc -c -o $# $<
The following command will compile file1.c, file2.c and file3.c in parallel. Then link stage will wait for the completion of these tree compilations.
make -j3
Directory discovery
The following command finds each Makefile and runs the make command:
find . -name Makefile -exec make -j3 -f '{}' ';' 2>&1 | tee result.txt
If you have 8 cores in your computer, you can use -j8.
Reuse in your main program
The build result is displayed on the shell screen and is also stored in the result.txt file.
More information?
I do not know what is your system, or your knowledge. Hope this can help. If you are not sure to understand some parts, please ask for more information. ;-)

generating a makefile for the dumb

I got 10 C files.
10 h files all in one folder.
I need those files to create 1 executable in the same folder using unix makefile.
EDIT :
the soultion
create a file named "makefile"
write the following make sure you have a single TAB before the word "gcc" this will create a.out executable
all:
gcc *.c
if you need flags just add them for example to make the filename BOB:
all:
gcc *.c -o BOB
I don't think you want what you say you want, but how about:
all:
gcc *.c
"missing separator" is commonly caused by a missing tab in front of a command line. The lines with $(CXX) need to be indented by a tab - not 8 spaces, not any number of spaces, but a tab.
Additionally, I don't think that empty lines between rule and commands are allowed.
Apart from obviously writing the Makefile yourself, you can also use CMake which is a convenient build system generator.
A simple example of a CMakeLists.txt file:
cmake_minimum_required(VERSION 2.6)
project(yourproject C)
add_executable(yourexecutable file1.c file1.h file2.c file2.h ...)
You can then do in a terminal:
$ cmake .
$ make
and your executable will be built.
Be careful however that the generated makefile uses cmake and is therefore not distributable per se.

Header Files in Multiple Directories: Best Practices

I'm a C Newb
I write lots of code in dynamic languages (javascript, python, haskell, etc.), but I'm now learning C for graduate school and I have no idea what I'm doing.
The Problem
Originally I was building all my source in one directory using a makefile, which has worked rather well. However, my project is growing and I would like to split the source into multiple directories (unit tests, utils, core, etc.). For example, my directory tree might look like the following:
.
|-- src
| |-- foo.c
| |-- foo.h
| `-- main.c
`-- test
`-- test_foo.c
test/test_foo.c uses both src/foo.c and src/foo.h. Using makefiles, what is the best/standard way to build this? Preferably, there would be one rule for building the project and one for building the tests.
Note
I know that there are other ways of doing this, including autoconf and other automatic solutions. However, I would like to understand what is happening and be able to write the makefiles from scratch despite its possible impracticality.
Any guidance or tips would be appreciated. Thanks!
[Edit]
So the three solutions given so far are as follows:
Place globally used header files in a parallel include directory
use the path in the #include satement as in #include "../src/foo.h"
use the -I switch to inform the compiler of include locations
So far I like the -I switch solution because it doesn't involve changing source code when directory structure changes.
For test_foo.c you simply need to tell the compiler where the header files can be found. E.g.
gcc -I../src -c test_foo.c
Then the compiler will also look into this directory to find the header files. In test_foo.c you write then:
#include "foo.h"
EDIT:
To link against foo.c, actually against foo.o, you need to mention it in the object file list. I assume you have already the object files, then do after that:
gcc test_foo.o ../src/foo.o -o test
I also rarely use the GNU autotools. Instead, I'll put a single hand-crafted makefile in the root directory.
To get all headers in the source directory, use something like this:
get_headers = $(wildcard $(1)/*.h)
headers := $(call get_headers,src)
Then, you can use the following to make the object-files in the test directory depend on these headers:
test/%.o : test/%.c $(headers)
gcc -std=c99 -pedantic -Wall -Wextra -Werror $(flags) -Isrc -g -c -o $# $<
As you can see, I'm no fan of built-in directives. Also note the -I switch.
Getting a list of object-files for a directory is slightly more complicated:
get_objects = $(patsubst %.c,%.o,$(wildcard $(1)/*.c))
test_objects = $(call get_objects,test)
The following rule would make the objects for your tests:
test : $(test_objects)
The test rule shouldn't just make the object files, but the executables. How to write the rule depends on the structure of your tests: Eg you could create an executable for each .c file or just a single one which tests all.
A common way of doing this is for header files used by a single C file to be named the same as that C file and in the same directory, and for header files used by many C files (especially those used by the whole project) to be in a directory include that is parallel to the C source directory.
Your test file should just include the header files directly using relative paths, like this:
#include "../src/foo.h"

Resources