I have a Haskell + stack + nix project that makes heavy use of FFI code. The problem is that two of the C files that I depend upon have to be generated before I can compile my Haskell project at large. These two files are (i) ./cbits/xdg-shell-protocol.c and (ii) ./include/xdg-shell-protocol.h.
First, here is the Makefile which can generate these files:
WAYLAND_PROTOCOLS=$(shell pkg-config --variable=pkgdatadir wayland-protocols)
WAYLAND_SCANNER=$(shell pkg-config --variable=wayland_scanner wayland-scanner)
# wayland-scanner is a tool which generates C headers and rigging for Wayland
# protocols, which are specified in XML. wlroots requires you to rig these up
# to your build system yourself and provide them in the include path.
xdg-shell-protocol.h:
$(WAYLAND_SCANNER) server-header \
$(WAYLAND_PROTOCOLS)/stable/xdg-shell/xdg-shell.xml $#
xdg-shell-protocol.c: xdg-shell-protocol.h
$(WAYLAND_SCANNER) private-code \
$(WAYLAND_PROTOCOLS)/stable/xdg-shell/xdg-shell.xml $#
Notice that I depend upon the system programs wayland-protocols and wayland-scanner, both of which are specified in my project's shell.nix:
buildInputs = with pkgs; [
# ...
# These are bleeding edge so I crafted my own nix expressions:
(callPackage ./nix/wayland.nix { } )
(callPackage ./nix/wayland-protocols.nix { } )
# ...
];
And finally notice that I tell Haskell about these files in my package.yaml:
c-sources:
- cbits/xdg-shell-protocol.c
include-dirs:
- include
Question: How can I make it so that everytime someone runs stack [--nix] build (with or without nix), the two files (i) ./cbits/xdg-shell-protocol.c and (ii) ./include/xdg-shell-protocol.h are ensured to be up to date?
Related
I am constructing an R package where I have few third party C functions which use GSL and OpenMP, and I then call these from R using wrappers made with Rcpp (which just modify some arguments and call the appropriate C function). Everything works fine in my Windows machine, but I am not sure how to define the Makevars.win and Makevars file in a portable way. My Makevars.win looks like this:
## This assumes that the LIB_GSL variable points to working GSL libraries
PKG_CPPFLAGS=-I$(LIB_GSL)/include -I../inst/include
PKG_LIBS=-L$(LIB_GSL)/lib/x64 -lgsl -lgslcblas $(SHLIB_OPENMP_CFLAGS)
PKG_CFLAGS=$(SHLIB_OPENMP_CFLAGS)
It is basically copied from various sources. Like said, this compiles in my computer (using toolchain of from Rtools), and if I remove PKG_CFLAGS=$(SHLIB_OPENMP_CFLAGS) I can also compile without OpenMP (for some reason I don't understand I get error if I remove OpenMP flag in PKG_LIBS).
My Makevars file looks identical, but I don't have access to Unix platforms so I am not sure how to deal with that side. My guess is that I need to replace LIB_GSL with something?
EDIT:
Okay, I think I finally understand how configure etc. works and was able to get everything working.
My Makevars.win:
## This assumes that the LIB_GSL variable points to working GSL libraries
PKG_CPPFLAGS=-I$(LIB_GSL)/include -I../inst/include
PKG_LIBS="-L$(LIB_GSL)/lib/$(R_ARCH)" -lgsl -lgslcblas $(SHLIB_OPENMP_CFLAGS)
PKG_CFLAGS=$(SHLIB_OPENMP_CFLAGS)
My Makevars.in:
GSL_CFLAGS = #GSL_CFLAGS#
GSL_LIBS = #GSL_LIBS#
PKG_LIBS=$(GSL_LIBS) $(SHLIB_OPENMP_CFLAGS)
PKG_CFLAGS=$(GSL_CFLAGS) $(SHLIB_OPENMP_CFLAGS)
And my configure.ac:
AC_INIT([testpackage], 0.0.1)
## Use gsl-config to find arguments for compiler and linker flags
##
## Check for non-standard programs: gsl-config(1)
AC_PATH_PROG([GSL_CONFIG], [gsl-config])
## If gsl-config was found, let's use it
if test "${GSL_CONFIG}" != ""; then
# Use gsl-config for header and linker arguments
GSL_CFLAGS=`${GSL_CONFIG} --cflags`
GSL_LIBS=`${GSL_CONFIG} --libs`
else
AC_MSG_ERROR([gsl-config not found, is GSL installed?])
fi
# Now substitute these variables in src/Makevars.in to create src/Makevars
AC_SUBST(GSL_CFLAGS)
AC_SUBST(GSL_LIBS)
AC_OUTPUT(src/Makevars)
I then run autoconf in testpackage directory in order to get configure file, which in turn converts Makevars.in to Makevars when running R CMD INSTALL.
There are a few CRAN packages using the GSL, and/or our RcppGSL bindings. Here is what I do in one of these (my RcppZiggurat package):
PKG_CPPFLAGS = -I. -I../inst/include
## Use the R_HOME indirection to support installations of multiple R version
PKG_LIBS = `$(R_HOME)/bin/Rscript -e "RcppGSL:::LdFlags()"`
That is the entire /src/Makevars. You can trivially add the same OpenMP variable from R. This does of course create a dependency on RcppGSL (as I already use it in RcppZiggurat). Else you can look into the R/init.R of RcppGSL and see how it tries to talk to gsl-config and store those values. You can do the same in src/Makevars -- it is just basic Make usage and nothing Rcpp specific.
I have a project that currently uses a communication library for communication. The library is not generic; it only runs on some machines. I want to port my project to be able to use a other libraries for communication. To do that, I defined a abstract layer that has different implementations. These implementations are consists of several files, so I want to group them into different folders.
For example, I might have a implementation that uses tcp, and another uses some system native library for communication. I have a folder tcp and a folder native in my src folder. I want the autotools to use one implementation that is selected at configure time with --with-comm=XXX.
How do I structure the Makefile.am to achieve this? I've read this: Different implementations in different files in autotools,
but that one uses only different files for different implementations, but I want to use folders.
You can use the AC_ARG_WITH macro in configure.ac, but it's more appropriate to use AC_ARG_ENABLE for internal, compile-time features. e.g., in configure.ac:
AC_ARG_ENABLE([comm], [AS_HELP_STRING([--enable-comm[[=tcp]]],
[specify a communications layer (default=tcp)])],
, [enable_comm=yes])
enable_comm=`echo $enable_comm` # strip whitespace.
if test "x$enable_comm" = "xyes" || "x$enable_comm" = "xtcp" ; then
enable_tcp="yes"
elif test "x$enable_comm" = "xnative" ; then
enable_native="yes"
elif test "x$enable_comm" = "xno" ; then
; # ... error message? default to a dummy layer implementation?
else
AC_MSG_ERROR([unknown option: '$enable_comm' for --enable-comm])
fi
AM_CONDITIONAL([ENABLE_COMM_TCP], [test "x$enable_tcp" = "xyes"])
AM_CONDITIONAL([ENABLE_COMM_NATIVE], [test "x$enable_native" = "xyes"])
...
AC_CONFIG_FILES([src/Makefile])
AC_CONFIG_FILES([src/tcp/Makefile src/native/Makefile])
...
AC_OUTPUT
You can, of course, put whatever you like in the help string, and set defaults differently.
In src/Makefile.am:
if ENABLE_COMM_TCP
COMM_DIR = tcp
endif
if ENABLE_COMM_NATIVE
COMM_DIR = native
endif
SUBDIRS = . $(COMM_DIR) # or '$(COMM_DIR) .' for depth-first order.
There are limitations because of the way automake works. It would be nice to be able to set COMM_DIR through a condition variable, but AFAIK this doesn't work. Furthermore, this approach automatically includes both tcp and native folders in the distribution if you use make dist. There's no need to put either in the EXTRA_DIST list, though it might be a better approach if the number of optional 'layers' becomes unwieldy.
If I understand your comments correctly, then the simple way would be to include the src files in the subdirectory build, so src/tcp/Makefile.am would use:
lib_LTLIBRARIES = libcomm.la
libcomm_la_SOURCES = driver.h driver.c ../comm.h ../wrapper.c ...
I can't remember offhand if comm.h needs to be specified as $(srcdir)/../comm.h for out-of-tree builds to work correctly. src/Makefile.am may or may not need to add comm.h, wrapper.c, etc., to its EXTRA_DIST variable. While this should work, it's not the 'right' way to go about things...
The library proper should be built in src, and that means using convenience libraries in either tcp or native, and letting libtool worry about the rest. e.g., src/Makefile:
lib_LTLIBRARIES = libcomm.la
libcomm_include_HEADERS = comm.h
libcomm_la_SOURCES = comm.h wrapper.c utils.c ...
libcomm_la_LIBADD = ./$(COMM_DIR)/libcomm_layer.la
It will also be necessary to order: SUBDIRS = $(COMM_DIR) .
In src/tcp/Makefile.am
AM_CPPFLAGS += -I$(srcdir)/.. # look for core headers in src
noinst_LTLIBRARIES = libcomm_layer.la # not for installation.
libcomm_layer_la_SOURCES = driver.h driver.c ...
# libcomm_layer_la_LDFLAGS = -static # optional... an archive is preferable.
This is a more future-proof approach as your 'core' and 'layer' libraries become more complex - and they will:)
A friend and I are using Qt Creator with Boost to build a game engine. So far we have this idea that the engine is going to be a shared library, with the idea that we can run it with a test executable which will turn into the game we eventually want to make.
The problem is header files, mainly. I'd like to find some way for Qt Creator to be able to recognize the header files as soon as the latest builds of the engine have been built or even when they're added. At first I was thinking a script in Python which executed as a build step in Qt Creator after the engine had been built, would simply copy the header files to a system directory (/usr/include, for example - if operating on a *nix system), where the IDE would then recognize the header files when linking the engine with the test executable, and we'd also have full auto completion support.
Of course, environmental variables would be used, and while I prefer developing in Linux, my friend prefers Windows, so we agreed to take care of development in regards to our respective platform preferences.
While this seems like a good solution, I think this Python script idea may be over kill. Is there a better way to do this?
Update
From to the suggested Qmake script, I end up getting this error.
cp -f "/home/amsterdam/Programming/atlas/Engine/AtlasEngine/"AtlasEngine_global.h "/"
cp: cannot create regular file `/AtlasEngine_global.h': Permission denied
make: Leaving directory `/home/amsterdam/Programming/atlas/Engine/AtlasEngine__GCC__Linux__Debug'
make: *** [libAtlasEngine.so.1.0.0] Error 1
15:20:52: The process "/usr/bin/make" exited with code 2.
Error while building project AtlasEngine (target: Desktop)
When executing build step 'Make'
My adjustments look as follows:
# Copy over build artifacts
SRCDIR = $$ATLAS_PROJ_ROOT
DESTDIR = $$ATLAS_INCLUDE
# Look for header files there too
INCLUDEPATH += $$SRCDIR
# Dependencies: mylib. Only specify the libs you depend on.
# Leave out for building a shared library without dependencies.
#win32:LIBS += $$quote($$SRCDIR/mylib.dll)
# unix:LIBS += $$quote(-L$$SRCDIR) -lmylib
DDIR = \"$$SRCDIR/\" #<--DEFAULTS
SDIR = \"$$IN_PWD/\"
# Replace slashes in paths with backslashes for Windows
win32:file ~= s,/,\\,g
win32:DDIR ~= s,/,\\,g
win32:SDIR ~= s,/,\\,g
for(file, HEADERS) {
QMAKE_POST_LINK += $$QMAKE_COPY $$quote($${SDIR}$${file}) $$quote($$DDIR) $$escape_expand(\\n\\t)
}
I have managed to overcome this using some Qmake magic that works cross-platform. It copies over the shared libraries (either .dll or .so files) along with the header files to a directory in a directory dll at a level next to your current project.
Put this in the end of your .pro files and change the paths/libs accordingly.
# Copy over build artifacts
MYDLLDIR = $$IN_PWD/../dlls
DESTDIR = \"$$MYDLLDIR\"
# Look for header files there too
INCLUDEPATH += $$MYDLLDIR
# Dependencies: mylib. Only specify the libs you depend on.
# Leave out for building a shared library without dependencies.
win32:LIBS += $$quote($$MYDLLDIR/mylib.dll)
unix:LIBS += $$quote(-L$$MYDLLDIR) -lmylib
DDIR = \"$$MYDLLDIR/\"
SDIR = \"$$IN_PWD/\"
# Replace slashes in paths with backslashes for Windows
win32:file ~= s,/,\\,g
win32:DDIR ~= s,/,\\,g
win32:SDIR ~= s,/,\\,g
for(file, HEADERS) {
QMAKE_POST_LINK += $$QMAKE_COPY $$quote($${SDIR}$${file}) $$quote($$DDIR) $$escape_expand(\\n\\t)
}
Then adjust the LD_LIBRARY_PATH in the 'Run settings' of your project to point to that same dll directory (relatively).
Yes, it's ugly with escaping for paths with spaces and backslashes, but I found this to be working well cross-platform. Windows (XP, 7) and Linux tested. And yes it includes environment settings to be changed for running your project, but at least you don't need external (Python) scripts anymore or to install it to system directory requiring root privileges.
Improvements are welcome.
I'm not sure if anyone else would be having issues with this, but for whatever reason Qmake wasn't able to access my user specified environment variables properly.
So, since this was the case, one solution I came up with was to add the variables as Qmake configuration variable.
If you're in a UNIX based system, the first thing you're going to want to do is append the location of qmake - which should lie in your QtSDK folder - to your system $PATH, like so:
export PATH=$PATH:/path/to/QtSDK/...../qmake_root
From there, you can do something along the lines of:
qmake -set "VARIABLE" "VALUE"
In this case, I simply did:
qmake -set "ATLAS_PROJ_ROOT" $ATLAS_PROJ_ROOT.
And then I accessed it in my Qmake project file (.pro) with:
VAR = $$[ATLAS_PROJ_ROOT]
More info can be found here.
I have a C project that has the following structure
Main/
Makefile.am
bin/
src/
Makefile.am
main.c
SomeLibrarySource/
SomeFuncs.c
SomeFuncs.h
The main.c contains the main function that uses functions defined in the SomeFuncs.{h/c} files.
I want to use autotools for this project. I read a couple of resources on autotools. But, I was only able to manage using autotools for a single level project where all source, object and other files reside in the same directory.
Then I got some links that talked about using autotools for deep projects like this one and then I got confused.
Right now I have two Makefile.am as follows
Makefile.am
SUBDIRS=src
src/Makefile.am
mainprgdir=../
mainprg_PROGRAMS=main
main_SOURCES=main.c
I am pretty sure that these files should not be as I have them now :P
How do I use autotools for the above project structure? (At least what should be there in those Makefile.am(s) and where should I place them.
EDIT:
One more thing! At the end I would like to have the object files created in the bin directory.
Thanks
mainprogdir=../ does not make a whole lot of sense (you don't know what it is relative to on installation). Probably intended:
# Main/Makefile.am
# .━━ target for `make install`
# |
# ↓ ↓━━ target for compilation
bin_PROGRAMS = bin/main
# ↓━━ based upon compilation target name
bin_main_SOURCES = src/main.c
There are two main approaches. If the functions in SomeLibrarySource are used only by main, then there's no need to build a separate library and you can simply specify the source files in src/Makefile.am
main_SOURCES = main.c SomeLibrarySource/SomeFuncs.c
However, if you actually want to use the functions in other code in your tree, you do not want to compile SomeFuncs.c multiple times but should use a convenience library.
# Assigning main_SOURCES is redundant
main_SOURCES = main.c
main_LDADD = SomeLibrarySource/libSomeFuncs.a
noinst_LIBRARIES = SomeLibrarySource/libSomeFuncs.a
AM_CPPFLAGS = -I$(srcdir)/SomeLibrarySource
(You'll need AC_PROG_RANLIB in configure.ac to use convenience libraries.)
If the source file is named SomeFuncs.c, automake will not need Makefile.am to specify SomeLibrarySource_libSomeFuncs_a_SOURCES, but if the name of the source code file does not match the name specified in noinst_LIBRARIES, SomeLibrarySource_libSomeFuncs_a_SOURCES should be set to the list of files used to build the library. Note that you do not need to specify main_SOURCES, since main.c is the default value if left unspecified (but it's not a bad idea to be explicit.) (In all of this, I am not comfortable use CamlCase names, but the system I'm using uses a case insensitive file system (biggest mistake apple ever made) and the examples I give here are working for me. YMMV)
You could of course do a recursive make, or build the library as a separate project and install it. (I like the final option. Libraries with useful features should exist on their own.)
I've got several directories with subdirectories containing c or asm files and I want them all compiled/assembled and then linked. I'm not especially picky where the object files go (e.g. a special bin folder or in the src folder) as long as a make clean removes them all.
The structure would look something like this:
/src
/dir1
/dir1_1
+file1_1.s
+file1_2.s
+file1.s
/dir2
+file2.c
I'm sure there's some easy way to create a makefile that compiles all files without me having to specify where it should look (compiling all files in one directory is doable with wildcards, but what then?).
Do a Google search for 'recursive make considered harmful'. You'll find the original article which postulates that the recursive make procedure is a bad way of doing business, and you'll find some links to other places which debate the validity of the proposition.
Basically, there are two ways to do builds in a directory hierarchy (with make).
Recursive make: each directory contains a makefile which builds in sub-directories and then builds in the current directory.
Non-recursive make: the makefile includes all the dependent makefiles, and builds up the complete dependency structure for the entire project and only builds the requisite software.
I work routinely on a product where the main build sequence is driven by a hybrid system that uses a shell script plus one makefile for each directory. One section of the product is managed by a 'RMCH' makefile; most of it is not. The build script deals with phases of the build, and sequences the directories, and runs make in each directory when it is time to do so. (The source code is in 20k+ files spread over a multitude of directories - it is a big project/product.)
I've also converted a medium-small project (about 20 directories of relevance, and about 400 source files) to work with RMCH (from a script + makefile-per-directory system). It was a bit mind-blowing at first, but works pretty neatly now it is done. Whether I did it correctly is open for debate; it was primarily a learning exercise, though I also did some work modifying the code to work with a modern curses library instead of the archaic BSD library that was used as a part of the code (archaic, as in 1982-vintage - the code was last seriously developed in about 1986) and generally upgrading to modern (standard C) standards. It was also a chance to work with git - so, all in all, quite an extensive learning experience.
If you can wrap your brain around RMCH, it is a good system. If done correctly, with complete and accurate dependency tracking, it removes the guess-work from the build sequence, and it does run fast. However, migrating even a medium size project to it is fairly hard work - it would be a daunting task to do it on the main product I work on, though the system might well benefit from it.
An alternative is to look at other alternatives to make, such as cmake, rake, scons, bras, imake, or ant or whatever else takes your fancy. Most of those are easily discoverable via a Google search; the hard one is bras, which is based on Tcl (as in Tcl/Tk), but is probably largely dead now. And imake is mentioned more for completeness than as a serious suggestion. You might also look at the GNU Autotools. Those do not abandon make; they build atop make.
If your project is small enough, you might get away with using a single hand-crafted makefile instead of a more sophisticated build system: check out the manual page on transformation functions to see what's possible.
Your example project could be compiled with the following non-recursive makefile:
targets = $(patsubst %$(1),%$(2),$(foreach dir,$(3),$(wildcard $(dir)/*$(1))))
asmdirs := src/dir1 src/dir1/dir1_1
cdirs := src/dir2
asmobjects := $(call targets,.s,.o,$(asmdirs))
cobjects := $(call targets,.c,.o,$(cdirs))
.PHONY : all clean
all : $(asmobjects) $(cobjects)
clean :
rm -f $(asmobjects) $(cobjects)
$(cobjects) : %.o : %.c
gcc -o $# -c $<
$(asmobjects) : %.o : %.s
gcc -o $# -c $<
However, because make can access the shell, you could also use standard unix tools like find instead of the somewhat limited builtin functions, eg
asmsources := $(shell find src -name '*.s')
csources := $(shell find src -name '*.c')
asmobjects := $(asmsources:.s=.o)
cobjects := $(csources:.c=.o)