I have a problem with a Makefile that's using an auto-generated Makefile to compile .o files from auto-generated .c and .s (assembler) source files.
The context is embedded programming for STM32 microcontrollers, where a Makefile and hardware initialization source code is generated from a hardware configuration file (.ioc file).
Suppose, we have the following directory structure:
.
├── archive
│ ├── a
│ │ └── a.c
│ ├── b
│ │ └── b.s
│ └── Makefile
├── Makefile
├── source
│ └── generate.sh
The files archive/a/a.c and archive/b/b.s may be empty here. My main ./Makefile looks like this:
.PHONY: default all clean generate objects
BUILD_DIR := build
SOURCE_DIR := source
OBJECTS := build/source/a/a.o build/source/b/b.o
default: all
all: objects
objects: $(OBJECTS)
$(BUILD_DIR)/%.o: %.c $(SOURCE_DIR)/Makefile
mkdir -p $(shell dirname $#)
$(MAKE) -C $(SOURCE_DIR) BUILD_DIR=../$(shell dirname $#) VPATH=../$(shell dirname $<) ../$#
$(BUILD_DIR)/%.o: %.s $(SOURCE_DIR)/Makefile
mkdir -p $(shell dirname $#)
$(MAKE) -C $(SOURCE_DIR) BUILD_DIR=../$(shell dirname $#) VPATH=../$(shell dirname $<) ../$#
$(SOURCE_DIR)/%.c :: generate
$(SOURCE_DIR)/%.s :: generate
$(SOURCE_DIR)/Makefile :: generate
generate: $(SOURCE_DIR)/generate.sh
cd $(SOURCE_DIR) ; bash generate.sh
clean:
$(RM) -r build $(SOURCE_DIR)/Makefile $(SOURCE_DIR)/a $(SOURCE_DIR)/b
The source/generate.sh looks like this:
cp -r ../archive/. ./
In reality, this process much more complicated and takes a few minutes.
Finally, the archive/Makefile looks like this:
BUILD_DIR := build
$(BUILD_DIR)/%.o: %.c
cp $^ $#
$(BUILD_DIR)/%.o: %.s
cp $^ $#
In reality, we use gcc instead of a simple cp to compile the C and ASM source files.
The auto-generated Makefile uses VPATH to find the required input file and is actually meant to compile everything in a flat build folder (which I don't want).
I have no control over the contents of the auto-generated Makefile! The source/generate.sh should be treated as a black-box that I have no influence on.
Therefore, the superordinate Makefile has to slightly abuse the auto-generated Makefile by overwriting the BUILD_DIR and VPATH accordingly.
Now, if I first run make generate and then, e.g. make build/source/a/a.o, everything works as planned:
First, the source directory is populated with the auto-generated stuff, and then build/source/a/a.o is "compiled" from source/a/a.c.
However, if I don't run make generate first, I get:
$ make build/source/a/a.o
make: *** No rule to make target 'build/source/a/a.o'. Stop.
My assumption was that the recipes would have been resolved in the following manner:
$(BUILD_DIR)/%.o: $(SOURCE_DIR)/%.c $(SOURCE_DIR)/Makefile
-> $(SOURCE_DIR)/%.c: generate
-> generate: $(SOURCE_DIR)/generate.sh
-> $(SOURCE_DIR)/Makefile: generate
-> generate: $(SOURCE_DIR)/generate.sh
After a bit of trial and error, I figured that I could somewhat fix this behaviour by defining the recipes as follows:
$(SOURCE_DIR)/%.c: generate
true
$(SOURCE_DIR)/%.s: generate
true
$(SOURCE_DIR)/Makefile: generate
true
This yields:
$ make build/source/a/a.o
cd source ; bash generate.sh
true
true
mkdir -p build/source/a
make -C source BUILD_DIR=../build/source/a VPATH=../source/a ../build/source/a/a.o
make[1]: Entering directory '/[...]/source'
cp ../source/a/a.c ../build/source/a/a.o
make[1]: Leaving directory '/[...]/source'
rm source/a/a.c
However, this leads to the generate step being run every time I run make build/source/a/a.o. Since the generate step is expensive in real life, this is not an option.
Also, source/a/a.c is treated as a temporary file that is meant to be deleted which I don't want.
How can I design my superordinate ./Makefile so that the generate step is being run automatically, but only if necessary?
I also need to be able to build non-auto-generated source files from the root directory.
A few more strange make behaviours:
Simply running make (i.e. make objects) with this Makefile yields
cd source ; bash generate.sh
true
true
mkdir -p build/source/a
make -C source BUILD_DIR=../build/source/a VPATH=../source/a ../build/source/a/a.o
make[1]: Entering directory '/[...]/source'
cp ../source/a/a.c ../build/source/a/a.o
make[1]: Leaving directory '/[...]/source'
true
mkdir -p build/source/b
make -C source BUILD_DIR=../build/source/b VPATH=../source/b ../build/source/b/b.o
make[1]: Entering directory '/[...]/source'
cp ../source/b/b.s ../build/source/b/b.o
make[1]: Leaving directory '/[...]/source'
rm source/a/a.c
Why does make remove source/a/a.c, but not source/b/b.c? For the record: I want neither of these files to be auto-removed.
An issue that I am sadly unable to reproduce with my example is that make basically outputs the following line at the start (adpated to the example):
make: Circular source/generate.sh.c <- generate dependency dropped.
Changing the recipe to the following
$(SOURCE_DIR)/%.c: generate
true
$(SOURCE_DIR)/%.s: generate
true
$(SOURCE_DIR)/Makefile: generate
yields:
$ make build/source/a/a.o
cd source ; bash generate.sh
true
cc -c -o source/Makefile.o source/Makefile.c
cc1: fatal error: source/Makefile.c: No such file or directory
compilation terminated.
make: *** [<builtin>: source/Makefile.o] Error 1
Why does make think that it needs to build a Makefile.o here?
I am currently trying to develop C libraries and project templates for the STM8 microcontroller using the SDCC OS compiler. My target is a (nearly) NOOB-compatible setup similar to Arduino - but with make+shellscripts instead of an IDE (there are limits to my ambition...)
Currently I am struggling with make to auto-detect dependencies. In Arduino the user only includes the relevant headers, e.g. "#include LCD-lib", and the build mechanism automatically detects dependency and links the respective libs. No need to manually add it to an IDE or the Makefile.
I love the simplicity of that, but so far I have failed miserably in creating a respective Makefile. Basically here's what the Makefile should achieve:
scan the *.c files in the project root for included headers. Note that these files are located in a different lib folder
add all included headers and (if exist) the corresponding C-files to the build process
to minimize compile time and size, unused C files in the lib folder must be skipped during build
I am confident that make can do all the above - but not at my level of experience with make... :-(
Here's the folder structure I have in mind:
├── Library
│ ├── Base
│ │ ├── general STM8 sources and headers
│ ├── STM8S_Discovery
│ │ └── board specific sources and headers
│ └── User
│ └── optional user library sources and headers
├── Projects
│ ├── Examples (to be filled)
│ │ └── Basic_Project
│ │ ├── compile_upload.sh --> double-click to build and upload
│ │ ├── config.h
│ │ ├── main.c
│ │ └── Makefile --> should detect dependencies in ./*.c and ./*.h
│ └── User_Projects (still empty)
└── Tools
├── programmer.py --> for programming (already works from make)
└── terminal.py --> for serial terminal (already works from make)
I know it's a lot to ask, but a convenient Makefile is my main blocking point. Any help is highly appreciated!!! Thanks a lot in advance!
Regards,
Georg Icking-Konert
Note: I realize that this answer doesn't meet all of your requirements, in fact this approach still requires you to list the names of relevant Arduino Libraries that you use in your project, as well as a list of paths to directories that should be included in the project. However, this solution is the closest to your requirements that I could think of and it might still help someone else reading this question down the road.
I use Arduino Makefile for this:
put Makefile.master in your main work-space directory
when you start a new Arduino project, you create it as a sub-directory in your workspace
create a single file with .pde/.ino extension containing setup() and `loop() methods
put the remaining logic into .c/.cpp/.h/.hpp files
add a project Makefile that sets project-refined settings in this sub-directory, e.g.:
# Your Arduino environment.
ARD_HOME = /usr/share/arduino
ARD_BIN = $(ARD_HOME)/hardware/tools/avr/bin
# Monitor Baudrate
MON_SPEED = 4800
# Board settings.
BOARD = uno
PORT = /dev/ttyACM0
PROGRAMMER = stk500v2
# Where to find header files and libraries.
INC_DIRS =
MY_LIB_DIRS =
LIBS =
LIB_DIRS = $(addprefix $(ARD_HOME)/libraries/, $(LIBS)) $(MY_LIB_DIRS)
include ../Makefile.master
compile and run using make all, make upload, make monitor, etc.
Ensure that you have picocom installed on your Unix/Linux machine (or equivalent) as console serial monitor. On MAC-OS, you can use screen by setting the MON_CMD variable accordingly.
Makefile.master:
The original Makefile.master was written by Alan Burlison and modified by Matthieu Weber, and can be found here.
I made some changes so that it fits my configuration, in particular I've added these lines of code:
### DEBUG Compilation ###
ifeq ($(DEBUG), 1)
ARD_FLAGS += -DDEBUG_PROJ
C_FLAGS += -g
CXX_FLAGS += -g
else
ARD_FLAGS += -DNDEBUG_PROJ
endif
and subsequently removed -g option from default C/CXX _FLAGS entries in Makefile.master. In this way symbol information is not added on release code, and only when code is compiled with DEBUG=1 the code shielded by
#ifdef DEBUG_PROJ
/* debug code here */
#endif
// or
#ifndef NDEBUG_PROJ
/* debug code here */
#endif
finds its way into the binary, thus resulting smaller release executables.
Here you can find my own version of the Makefile.master:
#
# Copyright 2011 Alan Burlison, alan#bleaklow.com. All rights reserved.
# Subsequently modified by Matthieu Weber, matthieu.weber#jyu.fi.
# Subsequently modified by Patrick Trentin, patrick.trentin.88#gmail.com
# Use is subject to license terms.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright notice,
# this list of conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above copyright notice,
# this list of conditions and the following disclaimer in the documentation
# and/or other materials provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY ALAN BURLISON "AS IS" AND ANY EXPRESS OR IMPLIED
# WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
# MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO
# EVENT SHALL ALAN BURLISON OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA,
# OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
# LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE,
# EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#
# Makefile for building Arduino projects outside of the Arduino environment
#
# This makefile should be included into a per-project Makefile of the following
# form:
#
# ----------
# BOARD = mega
# PORT = /dev/term/0
# INC_DIRS = ../common
# LIB_DIRS = ../libraries/Task ../../libraries/VirtualWire
# include ../../Makefile.master
# ----------
#
# Where:
# BOARD : Arduino board type, from $(ARD_HOME)/hardware/boards.txt
# PORT : USB port
# INC_DIRS : List pf directories containing header files
# LIB_DIRS : List of directories containing library source
#
# Before using this Makefile you can adjust the following macros to suit
# your environment, either by editing this file directly or by defining them in
# the Makefile that includes this one, in which case they will override the
# definitions below:
# ARD_REV : arduino software revision, e.g. 0017, 0018
# ARD_HOME : installation directory of the Arduino software.
# ARD_BIN : location of compiler binaries
# AVRDUDE : location of avrdude executable
# AVRDUDE_CONF : location of avrdude configuration file
# PROGRAMMER : avrdude programmer type
# MON_TERM : terminal command for serial monitor
# MON_CMD : serial monitor command
# MON_SPEED : serial monitor speed
#
# Global configuration.
ARD_REV ?= 100
ARD_HOME ?= /usr/local/arduino
ARD_BIN ?= /usr/bin
AVRDUDE ?= $(ARD_HOME)/hardware/tools/avrdude
AVRDUDE_CONF ?= $(ARD_HOME)/hardware/tools/avrdude.conf
MON_TERM ?= xterm
MON_SPEED ?= 57600
MON_CMD ?= picocom
PORT ?= $(HOME)/dev/arduino
BOARD ?= atmega328
### Nothing below here should require editing. ###
# Check for the required definitions.
ifndef BOARD
$(error $$(BOARD) not defined)
endif
ifndef PORT
$(error $$(PORT) not defined)
endif
# Version-specific settings
ARD_BOARDS = $(ARD_HOME)/hardware/arduino/boards.txt
ARD_SRC_DIR = $(ARD_HOME)/hardware/arduino/cores/arduino
ARD_MAIN = $(ARD_SRC_DIR)/main.cpp
# Standard macros.
SKETCH = $(notdir $(CURDIR))
BUILD_DIR = build
VPATH = $(LIB_DIRS)
# Macros derived from boards.txt
MCU := $(shell sed -n 's/$(BOARD)\.build\.mcu=\(.*\)/\1/p' < $(ARD_BOARDS))
F_CPU := $(shell sed -n 's/$(BOARD)\.build\.f_cpu=\(.*\)/\1/p' < $(ARD_BOARDS))
UPLOAD_SPEED := \
$(shell sed -n 's/$(BOARD)\.upload\.speed=\(.*\)/\1/p' < $(ARD_BOARDS))
PROGRAMMER := \
$(shell sed -n 's/$(BOARD)\.upload\.protocol=\(.*\)/\1/p' < $(ARD_BOARDS))
ARD_VAR := \
$(shell sed -n 's/$(BOARD)\.build\.variant=\(.*\)/\1/p' < $(ARD_BOARDS))
# More Version-specific settings
ARD_VAR_DIR = $(ARD_HOME)/hardware/arduino/variants/$(ARD_VAR)
# Build tools.
CC = $(ARD_BIN)/avr-gcc
CXX = $(ARD_BIN)/avr-g++
CXXFILT = $(ARD_BIN)/avr-c++filt
OBJCOPY = $(ARD_BIN)/avr-objcopy
OBJDUMP = $(ARD_BIN)/avr-objdump
AR = $(ARD_BIN)/avr-ar
SIZE = $(ARD_BIN)/avr-size
NM = $(ARD_BIN)/avr-nm
MKDIR = mkdir -p
RM = rm -rf
MV = mv -f
LN = ln -f
# Compiler flags.
INC_FLAGS = \
$(addprefix -I,$(INC_DIRS)) $(addprefix -I,$(LIB_DIRS)) -I$(ARD_SRC_DIR) -I$(ARD_VAR_DIR)
ARD_FLAGS = -mmcu=$(MCU) -DF_CPU=$(F_CPU) -DARDUINO=$(ARD_REV)
C_CXX_FLAGS = \
-Wall -Wextra -Wundef -Wno-unused-parameter \
-fdiagnostics-show-option -Wa,-adhlns=$(BUILD_DIR)/$*.lst
C_FLAGS = \
$(C_CXX_FLAGS) -std=gnu99 -Wstrict-prototypes -Wno-old-style-declaration
CXX_FLAGS = $(C_CXX_FLAGS)
### DEBUG Compilation ###
ifeq ($(DEBUG), 1)
ARD_FLAGS += -DDEBUG_PROJ
C_FLAGS += -g
CXX_FLAGS += -g
else
ARD_FLAGS += -DNDEBUG_PROJ
endif
# Optimiser flags.
# optimise for size, unsigned by default, pack data.
# separate sections, drop unused ones, shorten branches, jumps.
# don't inline, vectorise loops. no exceptions.
# no os preamble, use function calls in prologues.
# http://gcc.gnu.org/onlinedocs/gcc-4.3.5/gcc/
# http://www.tty1.net/blog/2008-04-29-avr-gcc-optimisations_en.html
OPT_FLAGS = \
-Os -funsigned-char -funsigned-bitfields -fpack-struct -fshort-enums \
-ffunction-sections -fdata-sections -Wl,--gc-sections,--relax \
-fno-inline-small-functions -fno-tree-scev-cprop -fno-exceptions \
-ffreestanding -mcall-prologues
# Build parameters.
IMAGE = $(BUILD_DIR)/$(SKETCH)
ARD_C_SRC = $(wildcard $(ARD_SRC_DIR)/*.c)
ARD_CXX_SRC = $(wildcard $(ARD_SRC_DIR)/*.cpp)
ARD_C_OBJ = $(patsubst %.c,%.o,$(notdir $(ARD_C_SRC)))
ARD_CXX_OBJ = $(patsubst %.cpp,%.o,$(notdir $(ARD_CXX_SRC)))
ARD_LIB = arduino
ARD_AR = $(BUILD_DIR)/lib$(ARD_LIB).a
ARD_AR_OBJ = $(ARD_AR)($(ARD_C_OBJ) $(ARD_CXX_OBJ))
ARD_LD_FLAG = -l$(ARD_LIB)
# Workaround for http://gcc.gnu.org/bugzilla/show_bug.cgi?id=34734
$(ARD_AR)(Tone.o) : CXX_FLAGS += -w
# Sketch libraries.
LIB_C_SRC = $(foreach ld,$(LIB_DIRS),$(wildcard $(ld)/*.c))
LIB_CXX_SRC = $(foreach ld,$(LIB_DIRS),$(wildcard $(ld)/*.cpp))
LIB_SRC = $(LIB_C_SRC) $(LIB_CXX_SRC)
ifneq "$(strip $(LIB_C_SRC) $(LIB_CXX_SRC))" ""
LIB_C_OBJ = $(patsubst %.c,%.o,$(notdir $(LIB_C_SRC)))
LIB_CXX_OBJ = $(patsubst %.cpp,%.o,$(notdir $(LIB_CXX_SRC)))
LIB_LIB = library
LIB_AR = $(BUILD_DIR)/lib$(LIB_LIB).a
LIB_AR_OBJ = $(LIB_AR)($(LIB_C_OBJ) $(LIB_CXX_OBJ))
LIB_LD_FLAG = -l$(LIB_LIB)
endif
# Sketch PDE source.
SKT_PDE_SRC = $(wildcard *.pde *.ino)
ifneq "$(strip $(SKT_PDE_SRC))" ""
SKT_PDE_OBJ = $(BUILD_DIR)/$(SKETCH)_pde.o
endif
# C and C++ source.
SKT_C_SRC = $(wildcard *.c)
SKT_CXX_SRC = $(wildcard *.cpp)
ifneq "$(strip $(SKT_C_SRC) $(SKT_CXX_SRC))" ""
SKT_C_OBJ = $(patsubst %.c,%.o,$(SKT_C_SRC))
SKT_CXX_OBJ = $(patsubst %.cpp,%.o,$(SKT_CXX_SRC))
SKT_LIB = sketch
SKT_AR = $(BUILD_DIR)/lib$(SKT_LIB).a
SKT_AR_OBJ = $(SKT_AR)/($(SKT_C_OBJ) $(SKT_CXX_OBJ))
SKT_LD_FLAG = -l$(SKT_LIB)
endif
# Definitions.
define run-cc
# $(CC) $(ARD_FLAGS) $(INC_FLAGS) -M -MT '$#($%)' -MF $#_$*.dep $<
$(CC) -c $(C_FLAGS) $(OPT_FLAGS) $(ARD_FLAGS) $(INC_FLAGS) \
$< -o $(BUILD_DIR)/$%
# $(AR) rc $# $(BUILD_DIR)/$%
# $(RM) $(BUILD_DIR)/$%
# $(CXXFILT) < $(BUILD_DIR)/$*.lst > $(BUILD_DIR)/$*.lst.tmp
# $(MV) $(BUILD_DIR)/$*.lst.tmp $(BUILD_DIR)/$*.lst
endef
define run-cxx
# $(CXX) $(ARD_FLAGS) $(INC_FLAGS) -M -MT '$#($%)' -MF $#_$*.dep $<
$(CXX) -c $(CXX_FLAGS) $(OPT_FLAGS) $(ARD_FLAGS) $(INC_FLAGS) \
$< -o $(BUILD_DIR)/$%
# $(AR) rc $# $(BUILD_DIR)/$%
# $(RM) $(BUILD_DIR)/$%
# $(CXXFILT) < $(BUILD_DIR)/$*.lst > $(BUILD_DIR)/$*.lst.tmp
# $(MV) $(BUILD_DIR)/$*.lst.tmp $(BUILD_DIR)/$*.lst
endef
# Rules.
.PHONY : all clean upload monitor upload_monitor
all : $(BUILD_DIR) $(IMAGE).hex
clean :
$(RM) $(BUILD_DIR)
$(BUILD_DIR) :
$(MKDIR) $#
$(SKT_PDE_OBJ) : $(SKT_PDE_SRC)
if [ $(ARD_REV) -ge 100 ]; then \
echo '#include "Arduino.h"' > $(BUILD_DIR)/$(SKETCH)_pde.cpp; \
else \
echo '#include "WProgram.h"' > $(BUILD_DIR)/$(SKETCH)_pde.cpp; \
fi
echo '#include "$(SKT_PDE_SRC)"' >> $(BUILD_DIR)/$(SKETCH)_pde.cpp
$(LN) $(SKT_PDE_SRC) $(BUILD_DIR)/$(SKT_PDE_SRC)
cd $(BUILD_DIR) && $(CXX) -c $(subst build/,,$(CXX_FLAGS)) \
$(OPT_FLAGS) $(ARD_FLAGS) -I.. \
$(patsubst -I..%,-I../..%,$(INC_FLAGS)) \
$(SKETCH)_pde.cpp -o $(#F)
(%.o) : $(ARD_SRC_DIR)/%.c
$(run-cc)
(%.o) : $(ARD_SRC_DIR)/%.cpp
$(run-cxx)
(%.o) : %.c
$(run-cc)
(%.o) : %.cpp
$(run-cxx)
$(BUILD_DIR)/%.d : %.c
$(run-cc-d)
$(BUILD_DIR)/%.d : %.cpp
$(run-cxx-d)
$(IMAGE).hex : $(ARD_AR_OBJ) $(LIB_AR_OBJ) $(SKT_AR_OBJ) $(SKT_PDE_OBJ)
$(CC) $(CXX_FLAGS) $(OPT_FLAGS) $(ARD_FLAGS) -L$(BUILD_DIR) \
$(SKT_PDE_OBJ) $(SKT_LD_FLAG) $(LIB_LD_FLAG) $(ARD_LD_FLAG) -lm \
-o $(IMAGE).elf
$(OBJCOPY) -O ihex -j .eeprom --set-section-flags=.eeprom=alloc,load \
--no-change-warnings --change-section-lma .eeprom=0 $(IMAGE).elf \
$(IMAGE).eep
$(OBJCOPY) -O ihex -R .eeprom $(IMAGE).elf $(IMAGE).hex
$(OBJDUMP) -h -S $(IMAGE).elf | $(CXXFILT) -t > $(IMAGE).lst
$(SIZE) $(IMAGE).hex
upload : all
- pkill -f '$(MON_CMD).*$(PORT)'
- sleep 1
- stty -F $(PORT) hupcl
- $(AVRDUDE) -V -C$(AVRDUDE_CONF) -p$(MCU) -c$(PROGRAMMER) -P$(PORT) \
-b$(UPLOAD_SPEED) -D -Uflash:w:$(IMAGE).hex:i
monitor :
LD_LIBRARY_PATH= LD_PRELOAD= \
$(MON_TERM) -title '$(BOARD) $(PORT)' \
-e '$(MON_CMD) -b $(MON_SPEED) $(PORT)' &
upload_monitor : upload monitor
-include $(wildcard $(BUILD_DIR)/*.dep))
# vim:ft=make
Use Example:
Given a dir-tree like follows:
Base_Dir
├── Library
│ ├── Base
│ │ ├── general STM8 sources and headers
│ ├── STM8S_Discovery
│ │ └── board specific sources and headers
│ └── User
│ └── optional user library sources and headers
├── Projects
│ ├── Examples (to be filled)
│ │ └── Basic_Project
│ │ ├── config.h
│ │ ├── example.ino
│ │ └── Makefile --> should detect dependencies in ./*.c and ./*.h
...
You could place Makefile.master within Projects, then assuming that:
You only need what is in Library/Base and Library/User for this project
You need to use LiquidCrystal Arduino Library in your project
then you would add the following Makefile into Basic Project:
# Your Arduino environment.
BASE_DIR = /path/to/Base_Dir # to edit
ARD_HOME = /usr/share/arduino # to edit, maybe
ARD_BIN = $(ARD_HOME)/hardware/tools/avr/bin
# Monitor Baudrate
MON_SPEED = 4800
# Board settings.
BOARD = uno
PORT = /dev/ttyACM0
PROGRAMMER = stk500v2
# Where to find header files and libraries.
INC_DIRS =
MY_LIB_DIRS= $(BASE_DIR)/Library/Base $(BASE_DIR)/Library/User
LIBS= LiquidCrystal
LIB_DIRS = $(addprefix $(ARD_HOME)/libraries/, $(LIBS)) $(MY_LIB_DIRS)
include ../../Makefile.master
Note that common.h should be automatically detected because it is located in ., and there should be no need to add the latter to INC_DIRS.
Final Note: last time I tested this configuration I was using version 1.0.5 of Arduino source code, and it was working flawlessly.
You can find a discussion, along with a sample implementation, of an automatic dependency generation method for GNU make here. You don't say you're using GNU make so I'm just assuming.
I don't know if that's sufficient for you or not; it wasn't clear from your statements about your requirements.
first of all thanks a lot to all for the fast and substantial support! I should've asked much earlier...
Now back to my (no longer an) issue. I now understand that I actually asked 2 different questions:
dependencies on headers -> solved by both Patrick's or MadScientist's proposal
detect what library is required by e.g. analysis of calls in main.o, or by the #includes in main.c...?
I understand the 2nd is much more difficult to achieve from within make...!? But Patrick's Makefile makes the manual configuration very convenient. So that's ok for me :-)
[add timeslip...] Ok, having pondered some more, would the following work / make sense?
call e.g. a gawk or python script from within Makefile to isolate all #included headers from the *.c and *.h in the project directory
for each included xyz.h check if a corresponding make_xyz exists in one of the Lib directories (would be part of the Lib)
if yes, include that makefile for compilation in the master Makefile
Does that somehow make sense...? Thanks again and have a nice day, wherever you are!
Regards, Georg Icking-Konert
thanks again for your support!
Following the above advice I wrote a small python script, which uses the gcc (or actually sdcc) dependency generator. The below script scans all project .c files for #included headers using gcc. The respective header files are then searched in the project and library folders. If a corresponding .c file exist (same path and name as header), it is added to the Makefile. This process is repeated until no more new headers are found.
The result is a Makefile which only builds modules which are #included modules in the project .c files - just as in the Arduino IDE. It may not be elegant but the job :-)
Lines 95-106 in the script are compiler and project specific, and have to be adapted accordingly. Have fun and thanks again!
#!/usr/bin/python
'''
automatically create a Makefile with dependencies from
all .c files in a starting directory
'''
# required modules
import sys
import os
import platform
import shlex
from subprocess import Popen, PIPE
# set OS specific
OS = platform.system()
if OS == 'Windows':
MAKE = 'mingw32-make.exe'
else:
MAKE = 'make'
##################
# helper functions
##################
#########
def getchar():
"""
python equivalent of getchar()
"""
ch = 0
if OS == 'Windows':
import msvcrt as m
ch = m.getch()
sys.stdio.flush()
sys.stderr.flush()
else:
import sys, tty, termios
fd = sys.stdin.fileno()
old_settings = termios.tcgetattr(fd)
tty.setraw(sys.stdin.fileno())
ch = sys.stdin.read(1)
return ch
# end getchar()
#########
def listFiles( start='.', pattern='.c' ):
"""
return set of matching files in project folder incl. subfolders
"""
result = set()
for root, dirs, files in os.walk(start):
for file in files:
if file.endswith(pattern):
#print(os.path.join(root, file))
result.add(os.path.join(root, file))
return result
# end listFiles()
#########
def listSubdirs( start='.' ):
"""
return set of subdirectories in given folder
"""
result = set()
for root, dirs, files in os.walk(start):
for dir in dirs:
#print(os.path.join(root, dir))
result.add(os.path.join(root, dir))
return result
# end listFiles()
#########
def get_exitcode_stdout_stderr(cmd):
"""
execute the external command and get its exitcode, stdout and stderr.
"""
args = shlex.split(cmd)
proc = Popen(args, stdout=PIPE, stderr=PIPE)
out, err = proc.communicate()
exitcode = proc.returncode
return exitcode, out, err
##################
# main program
##################
# set compile search paths
ROOT_DIR = '../../../'
TOOL_DIR = ROOT_DIR + 'Tools/'
LIB_ROOT = ROOT_DIR + 'Library/'
PRJ_ROOT = '.'
OBJDIR = 'output'
TARGET = 'main.ihx'
# set command for creating dependencies and set search paths
CC = 'sdcc '
CFLAGS = '-mstm8 --std-sdcc99 --std-c99 '
LFLAGS = '-mstm8 -lstm8 --out-fmt-ihx '
DEPEND = '-MM '
INCLUDE = '-I. '
for dir in listSubdirs(PRJ_ROOT):
INCLUDE += '-I' + dir + ' '
for dir in listSubdirs(LIB_ROOT):
INCLUDE += '-I' + dir + ' '
# get set of .c files in project folder incl. subdirectories
source_todo = listFiles(PRJ_ROOT,".c")
source_done = set()
header_done = set()
object_done = set()
# print message
sys.stdout.write('start Makefile creation ... ')
sys.stdout.flush()
# generate generic Makefile header
Makefile = open('Makefile', 'wb')
Makefile.write('OBJDIR = '+OBJDIR+'\n')
Makefile.write('TARGET = '+TARGET+'\n\n')
Makefile.write('.PHONY: clean all default objects\n\n')
Makefile.write('.PRECIOUS: $(TARGET)\n\n')
Makefile.write('default: $(OBJDIR) $(OBJDIR)/$(TARGET)\n\n')
Makefile.write('all: default\n\n')
Makefile.write('# create output folder\n')
Makefile.write('$(OBJDIR):\n')
Makefile.write(' mkdir -p $(OBJDIR)\n')
Makefile.write(' rm -fr -- -p\n\n')
# iteratively add project sources to Makefile
while (len(source_todo) > 0):
# get next pending source and mark as done
source = source_todo.pop()
source_done.add(source)
# convert Windows path to POSIX for Makefile
if OS == 'Windows':
source = source.replace('\\','/')
# use compiler generate dependency list
cmd = CC+DEPEND+CFLAGS+INCLUDE+source
#print cmd
exitcode, out, err = get_exitcode_stdout_stderr(cmd)
if (exitcode != 0):
print 'error: ' + err
getchar()
exit()
# append .c file with dependency and compile instruction to Makefile
Makefile.write('$(OBJDIR)/'+out)
#print(out)
Makefile.write('\t'+CC+CFLAGS+INCLUDE+'-c $< -o $#\n\n')
# extract file list including object[0], source[1] and headers[2..N]
out = out.replace(':', '')
out = out.replace('\\', '')
out = out.replace('\n', '')
out = out.split()
#print out
# for all files returned by compiler...
for next in out:
# append object files for linker
if next.endswith('.rel'):
object_done.add(next)
# if corresponding source to header exists, add to pending sources
if next.endswith('.h'):
if next not in header_done: # not yet in list
header_done.add(next) # add to treated headers
if (os.path.isfile(next[:-1]+'c')): # if corresponding .c exists, add to todo list
source_todo.add(next[:-1]+'c')
# link project object files
Makefile.write('$(OBJDIR)/$(TARGET): ')
for next in object_done:
Makefile.write('$(OBJDIR)/'+next+' ')
Makefile.write('\n')
Makefile.write('\t'+CC+LFLAGS)
for next in object_done:
Makefile.write('$(OBJDIR)/'+next+' ')
Makefile.write(' -o $#\n')
# close Makefile.dep
Makefile.close()
print('done\n')
sys.stdout.write('press any key to exit')
getchar()
exit()
# END OF MODULE
After reading the autotools mythbuster, I have tried to write a little example of the non recursive makefile with the use of subdir-objects in order to have my binary in the directory of my source files.
Here is the the organization of my little test:
/
autogen.sh
configure.ac
Makefile.am
src/
main.c
The autogen.sh :
#!/bin/sh
echo "Running aclocal..." ; aclocal $ACLOCAL_FLAGS || exit 1
echo "Running autoheader..." ; autoheader || exit 1
echo "Running autoconf..." ; autoconf || exit 1
echo "Running automake..." ; automake --add-missing --copy --gnu || exit 1
./configure "$#"
The configure.ac :
AC_PREREQ([2.69])
AC_INIT([test], [0.0.1], [devel#hell.org])
AC_CONFIG_SRCDIR([src/main.c])
AC_CONFIG_HEADERS([config.h])
# Checks for programs.
AC_PROG_CC
AM_INIT_AUTOMAKE([1.14 foreign subdir-objects])
AM_MAINTAINER_MODE([enable])
AC_OUTPUT(Makefile)
The Makefile.am :
MAINTAINERCLEANFILES = Makefile.in aclocal.m4 config.h.in configure depcomp install-sh missing compile
bin_PROGRAMS=toto
toto_SOURCES=src/main.c
I compile everything with :
./autogen.sh
make
I thought that the binary toto would have been created in the src directory with the option subdir-objects but it seems that this option have no effect and the toto binary is always generated in the root directory.
I have also tried to pass this option in the Makefile.am with AUTOMAKE_OPTIONS = subdir-objects but wihtout success.
That's not what the subdir-objects option does. It puts intermediate build results, especially *.o object files, in the same directory as the sources from which they are built. In particular, you should find main.o there instead of in the top directory.
The final build result, on the other hand, must be what and where you specify in your Automake file. If you want toto to go in the src/ directory too, then use this Makefile.am:
MAINTAINERCLEANFILES = Makefile.in aclocal.m4 config.h.in configure depcomp install-sh missing compile
bin_PROGRAMS=src/toto
src_toto_SOURCES=src/main.c
My source tree is
cpp
├── bin
├── CMakeLists.txt
├── src
├── headers
├── Makefile
├── TestConfig.h
├── TestConfig.h.in
├── README.txt
├── third_party
├── x_build
└── xtoolchain.cmake
I am using crosstool-ng.
Edit: Additional info (that may help others): I used reference of http://xecdesign.com/qemu-emulating-raspberry-pi-the-easy-way/ to boot RPI via QEMU (important resizing the img file to 8GB).
qemu-img resize 2014-01-07-wheezy-raspbian.img +6G
sudo fdisk /dev/mmcblk0
- use "parted" or 'd' options to delete the 2nd partition and then recreate it but with a larger size. (don't worry, the data will remain). 1st partition is vfat.
- reboot to activate the partition changes.
- use "sudo resize2fs /dev/mmclk0p2" (or /dev/sda2 if symlink set in udev rules) to enlarge the root file system.
- use e2fsck -f /dev/mmcblk0p2 to perform a file system check.
Installed all lib<package>-dev and other libraries using apt-get. Then shutdown the QEMU
mkdir -p /tmp/rasp
sudo mount /home/test/raspberrypi/2014-01-07-wheezy-raspbian.img /tmp/rasp/ -o offset=$((122880*512))
cd $HOME/raspberrypi
sudo cp -ar /tmp/rasp ./rootfs
sudo chown -R $USER:$USER rootfs/home/pi
cd $HOME/raspberrypi/rootfs
# also small hack around: error "Never use <bits/predefs.h> directly; include <stdc-predef.h> instead."
sudo cp /home/test/raspberrypi/rootfs/usr/include/features.h /home/test/raspberrypi/rootfs/usr/include/features.h-
sudo cp /home/test/local/x-tools/arm-unknown-linux-gnueabi/arm-unknown-linux-gnueabi/sysroot/usr/include/features.h /home/test/raspberrypi/rootfs/usr/include
sudo umount /tmp/rasp
I have also set up the target file system /home/test/raspberrypi/rootfs (This contains all the libraries necessary for cross compilation already installed)
The xtoolchain.cmake looks as follows
# ========================= CROSS COMPILATION INITIALIZATION BEGIN =======================
# this one is important
SET(CMAKE_SYSTEM_NAME Linux)
#this one not so much
SET(CMAKE_SYSTEM_VERSION 1)
# specify the cross compiler
SET(CMAKE_C_COMPILER
$ENV{HOME}/local/x-tools/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-gcc)
SET(CMAKE_CXX_COMPILER
$ENV{HOME}/local/x-tools/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-g++)
# where is the target environment
#SET(CMAKE_FIND_ROOT_PATH
#/home/test/local/x-tools/arm-unknown-linux-gnueabi /home/test/raspberrypi/rootfs)
SET(CMAKE_FIND_ROOT_PATH $ENV{HOME}/raspberrypi/rootfs)
# search for programs in the build host directories
SET(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
# for libraries and headers in the target directories
SET(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
SET(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
#include_directories ("/home/test/local/raspidev/include")
# ========================= CROSS COMPILATION INITIALIZATION END =========================
The CMakeLists.txt (This is in addition to CMakeLists.txt in the parent directory as you can see from the file structure) portion inside the src directory looks like
cmake_minimum_required(VERSION 2.8)
...
...
...
#Change 1 to 0 to disable logging
add_definitions(-DENABLE_LOG=1)
set(CMAKE_PREFIX_PATH $ENV{HOME}/raspberrypi/rootfs/)
set(CMAKE_LIBRARY_PATH $ENV{HOME}/raspberrypi/rootfs/lib:$ENV{HOME}/raspberrypi/rootfs/usr/lib:$ENV{HOME}/raspberrypi/rootfs/usr/local/lib)
set(CMAKE_MODULE_PATH $ENV{HOME}/raspberrypi/rootfs/usr/share/cmake-2.8/Modules)
#rdynamic is for stack unwinding on crash
set(CMAKE_CXX_FLAGS "-W -O3 -g -Wall -std=c++0x -rdynamic -D_AIBTRACE ${CMAKE_CXX_FLAGS}")
set_target_properties(${TARGET} PROPERTIES COMPILE_FLAGS ${CMAKE_CXX_FLAGS})
include_directories ("${PROJECT_SOURCE_DIR}/headers")
#add_subdirectory (headers)
...
...
...
# ======================== SQLITE3 =================================
pkg_check_modules(SQLITE3 REQUIRED sqlite3)
include_directories(${SQLITE3_INCLUDE_DIRS})
...
... (...And other libraries)
I do an out of source build inside the x_build directory. as follows
cmake -DCMAKE_TOOLCHAIN_FILE=/home/test/xcompile/cpp/xtoolchain.cmake ../
Strangely, it seems to be linking against the host libraries as opposed to the target libraries. make VERBOSE=1 shows this
It throws linker error for all the libraries (These libraries are definitely present on the target system rootfs)
/home/test/local/x-tools/arm-unknown-linux-gnueabi/lib/gcc/arm-unknown-linux-gnueabi/4.7.3/../../../../arm-unknown-linux-gnueabi/bin/ld: cannot find -lgmodule-2.0
/home/test/local/x-tools/arm-unknown-linux-gnueabi/lib/gcc/arm-unknown-linux-gnueabi/4.7.3/../../../../arm-unknown-linux-gnueabi/bin/ld: cannot find -lgthread-2.0
/home/test/local/x-tools/arm-unknown-linux-gnueabi/lib/gcc/arm-unknown-linux-gnueabi/4.7.3/../../../../arm-unknown-linux-gnueabi/bin/ld: cannot find -lxml2
/home/test/local/x-tools/arm-unknown-linux-gnueabi/lib/gcc/arm-unknown-linux-gnueabi/4.7.3/../../../../arm-unknown-linux-gnueabi/bin/ld: cannot find -lglib-2.0
/home/test/local/x-tools/arm-unknown-linux-gnueabi/lib/gcc/arm-unknown-linux-gnueabi/4.7.3/../../../../arm-unknown-linux-gnueabi/bin/ld: cannot find -lsqlite3
/home/test/local/x-tools/arm-unknown-linux-gnueabi/lib/gcc/arm-unknown-linux-gnueabi/4.7.3/../../../../arm-unknown-linux-gnueabi/bin/ld: cannot find -ldbus-1
collect2: error: ld returned 1 exit status
I am a bit lost.
Edit:
As requested here are few more things from CMakeLists.txt
set (SOURCES
file1.cc
file2.cc
main.cc
)
# Similarly, why not concise files grouping that will be reused ;)?
set (COMMONS_LIBS
${OpenCV_LIBS}
pthread
${GLIBMM_LIBRARIES}
${GTKMM_LIBRARIES}
#${GSTMM_LIBRARIES}
${GSTREAMER_LIBRARIES}
${SQLITE3_LIBRARIES}
#${DBUS_GLIB_LIBRARY}
${DBUS_LIBRARIES}
${DBUS_GLIB_LIBRARIES}
${OPENSSL_LIBRARIES}
)
add_executable (
${EXE_DAEMON}
${SOURCES}
)
target_link_libraries(
${EXE_DAEMON}
$COMMONS_LIBS}
)
install (TARGETS
${EXE_DAEMON}
DESTINATION bin)
I added the following to CMakeLists.txt and then few more changes below and got it working!
include($ENV{HOME}/raspberrypi/rootfs/usr/share/cmake-2.8/Modules/FindPkgConfig.cmake)
set(CMAKE_PREFIX_PATH $ENV{HOME}/raspberrypi/rootfs/)
set(SYSROOT $ENV{HOME}/raspberrypi/rootfs/)
set (PKG_CONFIG_SYSROOT_DIR ${SYSROOT})
set(PKG_CONFIG_PATH ${SYSROOT}/usr/lib/pkgconfig:${SYSROOT}/usr/local/lib/pkgconfig:${SYSROOT}/usr/lib/arm-linux-gnueabihf/pkgconfig:${SYSROOT}/usr/share/pkgconfig)
#Change 1 to 0 to disable logging
add_definitions(-DENABLE_LOG=1)
find_program(PKG_CONFIG_EXECUTABLE NAMES pkg-config PATHS /usr/bin DOC "pkg-config executable" NO_CMAKE_FIND_ROOT_PATH)
set (CMAKE_CXX_FLAGS "-W -O3 -g -Wall -std=c++0x -rdynamic -D_TRACE ${CMAKE_CXX_FLAGS}" CACHE STRING "" FORCE)
#rdynamic is for stack unwinding on crash
set(CMAKE_PREFIX_PATH $ENV{HOME}/raspberrypi/rootfs/)
set(CMAKE_LIBRARY_PATH $ENV{HOME}/raspberrypi/rootfs/lib:$ENV{HOME}/raspberrypi/rootfs/usr/lib:$ENV{HOME}/raspberrypi/rootfs/usr/local/lib)
set(CMAKE_MODULE_PATH $ENV{HOME}/raspberrypi/rootfs/usr/share/cmake-2.8/Modules)
include_directories(SYSTEM
$ENV{HOME}/raspberrypi/rootfs/usr/include/arm-linux-gnueabihf
$ENV{HOME}/raspberrypi/rootfs/usr/include
$ENV{HOME}/raspberrypi/rootfs/usr/local/include
)
add_definitions(
-march=armv6zk
-mfpu=vfp
-mfloat-abi=hard
)
link_directories(
$ENV{HOME}/raspberrypi/rootfs/lib/arm-linux-gnueabihf
$ENV{HOME}/raspberrypi/rootfs/lib
$ENV{HOME}/raspberrypi/rootfs/usr/lib/arm-linux-gnueabihf
$ENV{HOME}/raspberrypi/rootfs/usr/lib
$ENV{HOME}/raspberrypi/rootfs/usr/local/lib
)
set_target_properties(${TARGET} PROPERTIES COMPILE_FLAGS ${CMAKE_CXX_FLAGS})
Then did the following
sudo vim /home/test/raspberrypi/rootfs/usr/lib/arm-linux-gnueabihf/libpthread.so
edited
OUTPUT_FORMAT(elf32-littlearm)
GROUP ( /lib/arm-linux-gnueabihf/libpthread.so.0 /usr/lib/arm-linux-gnueabihf/libpthread_nonshared.a )
to
OUTPUT_FORMAT(elf32-littlearm)
GROUP ( libpthread.so.0 libpthread_nonshared.a )
similarly,
sudo vim /home/test/raspberrypi/rootfs/usr/lib/arm-linux-gnueabihf/libc.so
edited
OUTPUT_FORMAT(elf32-littlearm)
GROUP ( /lib/arm-linux-gnueabihf/libc.so.6 /usr/lib/arm-linux-gnueabihf/libc_nonshared.a AS_NEEDED ( /lib/arm-linux-gnueabihf/ld-linux-armhf.so.3 ) )
to
OUTPUT_FORMAT(elf32-littlearm)
GROUP ( libc.so.6 libc_nonshared.a AS_NEEDED ( ld-linux-armhf.so.3 ) )
Lastly, I added the -lpcre library to my COMMON_LIBS
Compile was successful. I then booted the raspberrypi via QEMU and used scp scp daemon pi#<ip address>:~ to transfer the resulting binary to PI emulator and ran it. Tadaa!!! it works fine!
I've just started to learn C (using Thinking In C) and I'm wondering about what files I should be ignoring in a C project's git repository.
No suggestion can be too obvious -- I'm a total noob. Thanks!
I guess there will be a few generated files that you don't wan't to be sticking in your repo (assuming your build output dir is in your git heirachy):
object files (.o, o.obj)
libraries (.lib)
DLLs, shared objects (.so, .dll)
Executables (.exe, a.out ?)
GIT ignore files are something I tend to do iteratively. "Hey, I don't need those things in my repo" ...
Edit: re dmckee's comment
Yep, you definately want to be ignoring swap files, temp files etc. I have the following as a baseline for my .gitignore:
*.swp
.~
thumbs.db
You can also setup your build to happen in a subdirectory say build and then you can ignore the whole thing inside .gitignore
build/
And you're done.
I use this in my .gitignore
But I am building for micro-controllers, so I don't know if it helps you much.
The easiest way to know, is just do a make clean, then add all your files, then do a make all and see what extra stuff appears.
#Some of these are related to eclipse. So i keep them out of my repo
.cproject
.dep/
.project
.settings/
#files being edited
*~
# make and build files
*.lst
*.o
*.eep
*.lss
*.map
*.sym
# I keep these, since I prefer having the reference of the final build
# *.elf
# *.hex
Github's .gitignore file templates cover most of the common files for projects in a variety of languages.
The C .gitignore template looks like this:
# Prerequisites
*.d
# Object files
*.o
*.ko
*.obj
*.elf
# Linker output
*.ilk
*.map
*.exp
# Precompiled Headers
*.gch
*.pch
# Libraries
*.lib
*.a
*.la
*.lo
# Shared objects (inc. Windows DLLs)
*.dll
*.so
*.so.*
*.dylib
# Executables
*.exe
*.out
*.app
*.i*86
*.x86_64
*.hex
# Debug files
*.dSYM/
*.su
*.idb
*.pdb
# Kernel Module Compile Results
*.mod*
*.cmd
.tmp_versions/
modules.order
Module.symvers
Mkfile.old
dkms.conf
Using a *nix system and a Makefile, you could add each generated file to .gitignore.
As an example I use the following when creating an executable from a single source (Example for a C executable generation):
%: %.c
gcc -o $# $<
grep '^$#$$' .gitignore > /dev/null || echo '$#' >> .gitignore
The following line can be added to other recipes to add target $# to .gitignore file:
grep '^$#$$' .gitignore > /dev/null || echo '$#' >> .gitignore
Explanation:
grep '^$#$$' .gitignore : searches for the target in .gitignore
^ indicates start of line
$$ is a single $ (but Makefile needs $$ to work) and indicates the end of the line
'^$#$$' represents the target name
|| : executes the next command only if the left hand one failed
so only if grep ... does not find the target name in .gitignore, echo ... is executed
echo '$#' >> .gitignore: adds the target name to .gitignore
Eventually, you will add to clean and rebuild everything to make sure all files are correctly ignored