I have a C project with the following structure with 1 target (binary final product)
main.c
configure.in
configure
Makefile.am
Makefile.in
folder-1
..Makefile.am
..Makefile.in
..<static library files .c files>
..<static library files .h files>
folder-2
<some .c files>
<some .h files>
...
...
I am aware how to configure and compile my project with Autotools. In regard to my library of folder-1: i am often changing files in that library with different debug levels by defining a flag called DMYDEBUG.
Compilation time for the whole project takes a while and by now, i am able to change the flag by
(1) modifiying the top-level configure.in file:
CCONFIGFLAGS="${CCONFIGFLAGS} -DSF_BIGENDIAN -DMYDEBUG=3"
(2) running make clean
(3) regenerating configure from the edited configure.in where i modify DMYDEBUG
(3) running ./configure on top level
(4) running make
only this way the wished effect is taking places. Is there a better way to modify DMYDEBUG (which is only relevant to the static library in folder-1) without having to recompile the whole project each time?
In the first place, it's terrible that you modify your configure.in to change the flag value. It would be much better to make configure recognize a custom argument that conveys the information, such as --with-debug-level=x. The AC_ARG_WITH() macro serves this purpose.
However, if you have to reconfigure the project (re-run ./configure, with or without rebuilding it first) to change the flag, then changing the flag will always require a full rebuild. For more narrowly-scoped rebuilding, you need to rely on make detecting the flag modification and re-building the affected targets.
make recognizes only file-level dependencies, so that strategy relies on you putting the macro definition in a header file, which the files that use it #include. Since you're using Automake, you can rely on your build system to recognize header dependencies automatically, but you may need to perform one clean build to bootstrap that.
Related
CMake offers several ways to specify the source files for a target.
One is to use globbing (documentation), for example:
FILE(GLOB MY_SRCS dir/*)
Another method is to specify each file individually.
Which way is preferred? Globbing seems easy, but I heard it has some downsides.
Full disclosure: I originally preferred the globbing approach for its simplicity, but over the years I have come to recognise that explicitly listing the files is less error-prone for large, multi-developer projects.
Original answer:
The advantages to globbing are:
It's easy to add new files as they
are only listed in one place: on
disk. Not globbing creates
duplication.
Your CMakeLists.txt file will be
shorter. This is a big plus if you
have lots of files. Not globbing
causes you to lose the CMake logic
amongst huge lists of files.
The advantages of using hardcoded file lists are:
CMake will track the dependencies of a new file on disk correctly - if we use
glob then files not globbed first time round when you ran CMake will not get
picked up
You ensure that only files you want are added. Globbing may pick up stray
files that you do not want.
In order to work around the first issue, you can simply "touch" the CMakeLists.txt that does the glob, either by using the touch command or by writing the file with no changes. This will force CMake to re-run and pick up the new file.
To fix the second problem you can organize your code carefully into directories, which is what you probably do anyway. In the worst case, you can use the list(REMOVE_ITEM) command to clean up the globbed list of files:
file(GLOB to_remove file_to_remove.cpp)
list(REMOVE_ITEM list ${to_remove})
The only real situation where this can bite you is if you are using something like git-bisect to try older versions of your code in the same build directory. In that case, you may have to clean and compile more than necessary to ensure you get the right files in the list. This is such a corner case, and one where you already are on your toes, that it isn't really an issue.
The best way to specify sourcefiles in CMake is by listing them explicitly.
The creators of CMake themselves advise not to use globbing.
See: https://cmake.org/cmake/help/latest/command/file.html?highlight=glob#glob
(We do not recommend using GLOB to collect a list of source files from your source tree. If no CMakeLists.txt file changes when a source is added or removed then the generated build system cannot know when to ask CMake to regenerate.)
Of course, you might want to know what the downsides are - read on!
When Globbing Fails:
The big disadvantage to globbing is that creating/deleting files won't automatically update the build-system.
If you are the person adding the files, this may seem an acceptable trade-off, however this causes problems for other people building your code, they update the project from version-control, run build, then contact you, complaining that"the build's broken".
To make matters worse, the failure typically gives some linking error which doesn't give any hints to the cause of the problem and time is lost troubleshooting it.
In a project I worked on we started off globbing but got so many complaints when new files were added, that it was enough reason to explicitly list files instead of globbing.
This also breaks common git work-flows(git bisect and switching between feature branches).
So I couldn't recommend this, the problems it causes far outweigh the convenience, when someone can't build your software because of this, they may loose a lot of time to track down the issue or just give up.
And another note, Just remembering to touch CMakeLists.txt isn't always enough, with automated builds that use globbing, I had to run cmake before every build since files might have been added/removed since last building *.
Exceptions to the rule:
There are times where globbing is preferable:
For setting up a CMakeLists.txt files for existing projects that don't use CMake.Its a fast way to get all the source referenced (once the build system's running - replace globbing with explicit file-lists).
When CMake isn't used as the primary build-system, if for example you're using a project who aren't using CMake, and you would like to maintain your own build-system for it.
For any situation where the file list changes so often that it becomes impractical to maintain. In this case it could be useful, but then you have to accept running cmake to generate build-files every time to get a reliable/correct build (which goes against the intention of CMake - the ability to split configuration from building).
* Yes, I could have written a code to compare the tree of files on disk before and after an update, but this is not such a nice workaround and something better left up to the build-system.
In CMake 3.12, the file(GLOB ...) and file(GLOB_RECURSE ...) commands gained a CONFIGURE_DEPENDS option which reruns cmake if the glob's value changes.
As that was the primary disadvantage of globbing for source files, it is now okay to do so:
# Whenever this glob's value changes, cmake will rerun and update the build with the
# new/removed files.
file(GLOB_RECURSE sources CONFIGURE_DEPENDS "*.cpp")
add_executable(my_target ${sources})
However, some people still recommend avoiding globbing for sources. Indeed, the documentation states:
We do not recommend using GLOB to collect a list of source files from your source tree. ... The CONFIGURE_DEPENDS flag may not work reliably on all generators, or if a new generator is added in the future that cannot support it, projects using it will be stuck. Even if CONFIGURE_DEPENDS works reliably, there is still a cost to perform the check on every rebuild.
Personally, I consider the benefits of not having to manually manage the source file list to outweigh the possible drawbacks. If you do have to switch back to manually listed files, this can be easily achieved by just printing the globbed source list and pasting it back in.
You can safely glob (and probably should) at the cost of an additional file to hold the dependencies.
Add functions like these somewhere:
# Compare the new contents with the existing file, if it exists and is the
# same we don't want to trigger a make by changing its timestamp.
function(update_file path content)
set(old_content "")
if(EXISTS "${path}")
file(READ "${path}" old_content)
endif()
if(NOT old_content STREQUAL content)
file(WRITE "${path}" "${content}")
endif()
endfunction(update_file)
# Creates a file called CMakeDeps.cmake next to your CMakeLists.txt with
# the list of dependencies in it - this file should be treated as part of
# CMakeLists.txt (source controlled, etc.).
function(update_deps_file deps)
set(deps_file "CMakeDeps.cmake")
# Normalize the list so it's the same on every machine
list(REMOVE_DUPLICATES deps)
foreach(dep IN LISTS deps)
file(RELATIVE_PATH rel_dep ${CMAKE_CURRENT_SOURCE_DIR} ${dep})
list(APPEND rel_deps ${rel_dep})
endforeach(dep)
list(SORT rel_deps)
# Update the deps file
set(content "# generated by make process\nset(sources ${rel_deps})\n")
update_file(${deps_file} "${content}")
# Include the file so it's tracked as a generation dependency we don't
# need the content.
include(${deps_file})
endfunction(update_deps_file)
And then go globbing:
file(GLOB_RECURSE sources LIST_DIRECTORIES false *.h *.cpp)
update_deps_file("${sources}")
add_executable(test ${sources})
You're still carting around the explicit dependencies (and triggering all the automated builds!) like before, only it's in two files instead of one.
The only change in procedure is after you've created a new file. If you don't glob the workflow is to modify CMakeLists.txt from inside Visual Studio and rebuild, if you do glob you run cmake explicitly - or just touch CMakeLists.txt.
Specify each file individually!
I use a conventional CMakeLists.txt and a python script to update it. I run the python script manually after adding files.
See my answer here:
https://stackoverflow.com/a/48318388/3929196
I'm not a fan of globbing and never used it for my libraries. But recently I've looked a presentation by Robert Schumacher (vcpkg developer) where he recommends to treat all your library sources as separate components (for example, private sources (.cpp), public headers (.h), tests, examples - are all separate components) and use separate folders for all of them (similarly to how we use C++ namespaces for classes). In that case I think globbing makes sense, because it allows you to clearly express this components approach and stimulate other developers to follow it. For example, your library directory structure can be the following:
/include - for public headers
/src - for private headers and sources
/tests - for tests
You obviously want other developers to follow your convention (i.e., place public headers under /include and tests under /tests). file(glob) gives a hint for developers that all files from a directory have the same conceptual meaning and any files placed to this directory matching the regexp will also be treated in the same way (for example, installed during 'make install' if we speak about public headers).
I've got a tool that generates files that contain definitions and declarations. These files need to be included from other source files or headers - they aren't usable standalone.
The obvious thing to do is have a custom command to generate them. My CMakeLists.txt that does this is as follows. I'm currently using this with the GNU makefile generator.
project(test_didl)
cmake_minimum_required(VERSION 3.0)
add_custom_command(
OUTPUT test_didl_structs.h test_didl_structs.c
COMMAND python ${CMAKE_CURRENT_SOURCE_DIR}/didl.py --decls=test_didl_structs.h --defs=test_didl_structs.c ${CMAKE_CURRENT_SOURCE_DIR}/test_didl_structs.py
DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/didl.py ${CMAKE_CURRENT_SOURCE_DIR}/test_didl_structs.py
MAIN_DEPENDENCY ${CMAKE_CURRENT_SOURCE_DIR}/test_didl_structs.py)
add_executable(test_didl test_didl.c)
target_include_directories(test_didl PRIVATE ${CMAKE_CURRENT_BINARY_DIR})
target_link_libraries(test_didl shared_lib)
test_didl.c is very simple:
#include <stdint.h>
#include <stdio.h>
#include <string.h>
#include "test_didl_structs.h"
#include "test_didl_structs.c"
int main(void) {
}
But on the first build, make tries to build test_didl.c, which of course fails, because test_didl_structs.* haven't been generated yet. Naturally, before the first successful build of test_didl.c, the dependency information isn't known, so make doesn't know to run the python command first.
I tried a custom target, but that's no good, because custom targets are assumed to be always dirty. This means the C file is recompiled on every build and the EXE is linked. This approach won't scale.
My eventual solution was to make the output .h file an input to the executable:
add_executable(test_didl test_didl.c test_didl_structs.h)
.h file inputs are treated as dependencies, but don't otherwise do anything interesting for makefile generators. (I am not currently interested in other generators.)
So that works, but it feels a bit ugly. It doesn't actually state explicitly that the custom commands need to be run first, though in practice this seems to happen. I'm not quite sure how, though (but I'm not up to speed on reading the CMake-generated Makefiles just yet).
Is this how it's supposed to work? Or is there something neater I'm supposed to be doing instead?
(What I'm imagining, I suppose, is something like a Visual Studio pre-build step, in that it's considered for running on every build, before the normal dependency checking. But I want this pre-build step to have dependency checking, so that it's skipped if its inputs are older than its outputs.)
My eventual solution was to make the output .h file an input to the executable.
This way is correct.
It actually states, that building executable depends on given file, and, if that file is OUTPUT for some add_custom_command(), this command will be executed before building executable.
Another way is to generate needed headers at configuration stage using execute_process(). In that case there is no need to add header files as sources for add_executable(): CMake has notion of autodetecting dependencies for compiling, so test_didl will be rebuilt after regeneration of test_didl_structs.h.
execute_process(COMMAND python ${CMAKE_CURRENT_SOURCE_DIR}/didl.py --decls=test_didl_structs.h --defs=test_didl_structs.c ${CMAKE_CURRENT_SOURCE_DIR}/test_didl_structs.py)
# ...
add_executable(test_didl test_didl.c)
Drawback of this approach is that you need manually rerun configuration stage after changing your .py files. See also that question and answer to it.
Another problem is that header file will be updated every time configuration is run.
You can try tell cmake that you are using an external source, see docs about set_source_files_properties, see this past post
I'm an autotools beginner, and I can't seem to figure out how to use an external library correctly with autotools.
Here is my directory hierarchy.
.
├── configure.ac
├── Makefile.am
├── README
└── src
(source files)
└── Makefile.am
The library's name is acml_mp and is, by default, installed in /opt/acml5.3.1/gfortran64/lib. There is also a directory called /opt/acml5.3.1/gfortran64/include to include. When I compile without autotools, including the usual compiler flags works fine:
g++ ... -L/opt/acml5.3.1/gfortran64_mp/lib -I/opt/acml5.3.1/gfortran64_mp/include -lacml_mp ...
In configure.ac, I put the command AC_LIB_LINKFLAGS([acml_mp]) which I think only deals with the -lacml_mp flag.
Basically, the end goal is to have autoconf search for this library, and have the makefile generated by automake include the correct link/include paths when compiling. Finally, when compiling by hand, I always need to modify the environment variable LD_LIBRARY_PATH using the command
Export LD_LIBRARY_PATH=/opt/acml5.3.1/gfortran64_mp/lib:$LD_LIBRARY_PATH
which, ideally, I would like to avoid having the user do. Apologies if this information exists already, I looked through SO and Google for a few hours to no avail.
The problem with searching is that /opt/acml5.3.1/gfortran is never going to be a standard (search) location for libraries (and headers) like /usr/lib, /usr/local/lib etc. Probably the best bet is to supply this location explicitly via --with-acml to configure.
The AC_ARG_WITH macro is described here. Assuming test "x$with_acml" != xno, you can try linking a program with AC_LINK_IFELSE.
AC_LANG_PUSH([C]) # or [Fortran]
ac_save_acml_CPPFLAGS="$CPPFLAGS" # or FCFLAGS instead of CPPFLAGS.
ac_save_acml_LIBS="$LIBS"
ac_acml_CPPFLAGS="-I${with_acml}/include"
ac_acml_LIBS="-L${with_acml}/libs -lacml_mp"
CPPFLAGS+="$ac_acml_CPPFLAGS"
LIBS+="$ac_acml_LIBS"
AC_LINK_IFELSE([AC_LANG_PROGRAM( ... some C or Fortran program ... )],,
AC_MSG_FAILURE([couldn't link with acml]))
AC_LANG_POP
# we *could* stop here... but we might need the original values later.
CPPFLAGS="$ac_save_acml_CPPFLAGS"
LIBS="$ac_save_acml_LIBS"
AC_SUBST(ACML_CPPFLAGS, $ac_acml_CPPFLAGS)
AC_SUBST(ACML_LIBFLAGS, $ac_acml_LIBS)
Assuming you've initialized libtool support with LT_INIT, you can add the acml library with $(ACML_LIBFLAGS) to your own libraries in src/Makefile.am via the LIBADD variable, or to executables with the LDADD variable. or <lib>_la_LIBADD, <prog>_LDADD respectively.
To compile sources with the $(ACML_CPPFLAGS) include path, add it to the AM_CPPFLAGS variable. Or the <prog>_CPPFLAGS variable.
It's difficult to be specific without knowing how your Makefile.am is already set up. I know it looks complicated - but it's better to get the infrastructure right the first time. I'll add to the answer if you have further questions.
I'm trying to adjust 3rd person code to my needs. This code is provided with CMake config files used to build and install it. There is possibility to choose one of libraries. And in code is often used #ifdef USE_FTD2XX directive. I saw that this is defined in CMamkeFiles.txt file like here:
option(USE_FTD2XX "Use FTDI libFTD2XX instead of free libftdi" ON)
if(USE_FTD2XX)
find_package(libFTD2XX)
endif(USE_FTD2XX)
if(LIBFTD2XX_FOUND)
include_directories(${LIBFTD2XX_INCLUDE_DIR})
add_definitions( -DUSE_FTD2XX )
else(LIBFTD2XX_FOUND)
set(LIBFTD2XX_LIBRARIES "")
endif(LIBFTD2XX_FOUND)
But if I simply use *.c and *.cpp files and I analyse and run it simply from IDE (Codeblocks), how could I set using this library in C++ code instead of in CMake? I'm also sure that I want use always this one so it can be fixed.
Should I simply #define USE_FTD2XX in main file?
You cannot simply #define USE_FTD2XX because you also need specific linker options for this to work (i.e. the library to link with). If the option is OFF in cmake, the specific link options won't be present in the Makefile and most likely you'll have linker errors.
So CMake takes care of everything automatically for you, but you need to re-generate your makefiles each time you want to toggle options on/off.
If only headers were involved and no library to link with (like some parts of the Boost framework), then yeah, defining USE_FTD2XX in your should be enough.
I have a C project that has the following structure
Main/
Makefile.am
bin/
src/
Makefile.am
main.c
SomeLibrarySource/
SomeFuncs.c
SomeFuncs.h
The main.c contains the main function that uses functions defined in the SomeFuncs.{h/c} files.
I want to use autotools for this project. I read a couple of resources on autotools. But, I was only able to manage using autotools for a single level project where all source, object and other files reside in the same directory.
Then I got some links that talked about using autotools for deep projects like this one and then I got confused.
Right now I have two Makefile.am as follows
Makefile.am
SUBDIRS=src
src/Makefile.am
mainprgdir=../
mainprg_PROGRAMS=main
main_SOURCES=main.c
I am pretty sure that these files should not be as I have them now :P
How do I use autotools for the above project structure? (At least what should be there in those Makefile.am(s) and where should I place them.
EDIT:
One more thing! At the end I would like to have the object files created in the bin directory.
Thanks
mainprogdir=../ does not make a whole lot of sense (you don't know what it is relative to on installation). Probably intended:
# Main/Makefile.am
# .━━ target for `make install`
# |
# ↓ ↓━━ target for compilation
bin_PROGRAMS = bin/main
# ↓━━ based upon compilation target name
bin_main_SOURCES = src/main.c
There are two main approaches. If the functions in SomeLibrarySource are used only by main, then there's no need to build a separate library and you can simply specify the source files in src/Makefile.am
main_SOURCES = main.c SomeLibrarySource/SomeFuncs.c
However, if you actually want to use the functions in other code in your tree, you do not want to compile SomeFuncs.c multiple times but should use a convenience library.
# Assigning main_SOURCES is redundant
main_SOURCES = main.c
main_LDADD = SomeLibrarySource/libSomeFuncs.a
noinst_LIBRARIES = SomeLibrarySource/libSomeFuncs.a
AM_CPPFLAGS = -I$(srcdir)/SomeLibrarySource
(You'll need AC_PROG_RANLIB in configure.ac to use convenience libraries.)
If the source file is named SomeFuncs.c, automake will not need Makefile.am to specify SomeLibrarySource_libSomeFuncs_a_SOURCES, but if the name of the source code file does not match the name specified in noinst_LIBRARIES, SomeLibrarySource_libSomeFuncs_a_SOURCES should be set to the list of files used to build the library. Note that you do not need to specify main_SOURCES, since main.c is the default value if left unspecified (but it's not a bad idea to be explicit.) (In all of this, I am not comfortable use CamlCase names, but the system I'm using uses a case insensitive file system (biggest mistake apple ever made) and the examples I give here are working for me. YMMV)
You could of course do a recursive make, or build the library as a separate project and install it. (I like the final option. Libraries with useful features should exist on their own.)