Arduino-like Makefile with dependencies...? - c

I am currently trying to develop C libraries and project templates for the STM8 microcontroller using the SDCC OS compiler. My target is a (nearly) NOOB-compatible setup similar to Arduino - but with make+shellscripts instead of an IDE (there are limits to my ambition...)
Currently I am struggling with make to auto-detect dependencies. In Arduino the user only includes the relevant headers, e.g. "#include LCD-lib", and the build mechanism automatically detects dependency and links the respective libs. No need to manually add it to an IDE or the Makefile.
I love the simplicity of that, but so far I have failed miserably in creating a respective Makefile. Basically here's what the Makefile should achieve:
scan the *.c files in the project root for included headers. Note that these files are located in a different lib folder
add all included headers and (if exist) the corresponding C-files to the build process
to minimize compile time and size, unused C files in the lib folder must be skipped during build
I am confident that make can do all the above - but not at my level of experience with make... :-(
Here's the folder structure I have in mind:
├── Library
│   ├── Base
│   │   ├── general STM8 sources and headers
│   ├── STM8S_Discovery
│   │   └── board specific sources and headers
│   └── User
│      └── optional user library sources and headers
├── Projects
│   ├── Examples (to be filled)
│   │   └── Basic_Project
│   │   ├── compile_upload.sh --> double-click to build and upload
│   │   ├── config.h
│   │   ├── main.c
│   │   └── Makefile --> should detect dependencies in ./*.c and ./*.h
│   └── User_Projects (still empty)
└── Tools
├── programmer.py --> for programming (already works from make)
└── terminal.py --> for serial terminal (already works from make)
I know it's a lot to ask, but a convenient Makefile is my main blocking point. Any help is highly appreciated!!! Thanks a lot in advance!
Regards,
Georg Icking-Konert

Note: I realize that this answer doesn't meet all of your requirements, in fact this approach still requires you to list the names of relevant Arduino Libraries that you use in your project, as well as a list of paths to directories that should be included in the project. However, this solution is the closest to your requirements that I could think of and it might still help someone else reading this question down the road.
I use Arduino Makefile for this:
put Makefile.master in your main work-space directory
when you start a new Arduino project, you create it as a sub-directory in your workspace
create a single file with .pde/.ino extension containing setup() and `loop() methods
put the remaining logic into .c/.cpp/.h/.hpp files
add a project Makefile that sets project-refined settings in this sub-directory, e.g.:
# Your Arduino environment.
ARD_HOME = /usr/share/arduino
ARD_BIN = $(ARD_HOME)/hardware/tools/avr/bin
# Monitor Baudrate
MON_SPEED = 4800
# Board settings.
BOARD = uno
PORT = /dev/ttyACM0
PROGRAMMER = stk500v2
# Where to find header files and libraries.
INC_DIRS =
MY_LIB_DIRS =
LIBS =
LIB_DIRS = $(addprefix $(ARD_HOME)/libraries/, $(LIBS)) $(MY_LIB_DIRS)
include ../Makefile.master
compile and run using make all, make upload, make monitor, etc.
Ensure that you have picocom installed on your Unix/Linux machine (or equivalent) as console serial monitor. On MAC-OS, you can use screen by setting the MON_CMD variable accordingly.
Makefile.master:
The original Makefile.master was written by Alan Burlison and modified by Matthieu Weber, and can be found here.
I made some changes so that it fits my configuration, in particular I've added these lines of code:
### DEBUG Compilation ###
ifeq ($(DEBUG), 1)
ARD_FLAGS += -DDEBUG_PROJ
C_FLAGS += -g
CXX_FLAGS += -g
else
ARD_FLAGS += -DNDEBUG_PROJ
endif
and subsequently removed -g option from default C/CXX _FLAGS entries in Makefile.master. In this way symbol information is not added on release code, and only when code is compiled with DEBUG=1 the code shielded by
#ifdef DEBUG_PROJ
/* debug code here */
#endif
// or
#ifndef NDEBUG_PROJ
/* debug code here */
#endif
finds its way into the binary, thus resulting smaller release executables.
Here you can find my own version of the Makefile.master:
#
# Copyright 2011 Alan Burlison, alan#bleaklow.com. All rights reserved.
# Subsequently modified by Matthieu Weber, matthieu.weber#jyu.fi.
# Subsequently modified by Patrick Trentin, patrick.trentin.88#gmail.com
# Use is subject to license terms.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright notice,
# this list of conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above copyright notice,
# this list of conditions and the following disclaimer in the documentation
# and/or other materials provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY ALAN BURLISON "AS IS" AND ANY EXPRESS OR IMPLIED
# WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
# MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO
# EVENT SHALL ALAN BURLISON OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA,
# OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
# LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE,
# EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#
# Makefile for building Arduino projects outside of the Arduino environment
#
# This makefile should be included into a per-project Makefile of the following
# form:
#
# ----------
# BOARD = mega
# PORT = /dev/term/0
# INC_DIRS = ../common
# LIB_DIRS = ../libraries/Task ../../libraries/VirtualWire
# include ../../Makefile.master
# ----------
#
# Where:
# BOARD : Arduino board type, from $(ARD_HOME)/hardware/boards.txt
# PORT : USB port
# INC_DIRS : List pf directories containing header files
# LIB_DIRS : List of directories containing library source
#
# Before using this Makefile you can adjust the following macros to suit
# your environment, either by editing this file directly or by defining them in
# the Makefile that includes this one, in which case they will override the
# definitions below:
# ARD_REV : arduino software revision, e.g. 0017, 0018
# ARD_HOME : installation directory of the Arduino software.
# ARD_BIN : location of compiler binaries
# AVRDUDE : location of avrdude executable
# AVRDUDE_CONF : location of avrdude configuration file
# PROGRAMMER : avrdude programmer type
# MON_TERM : terminal command for serial monitor
# MON_CMD : serial monitor command
# MON_SPEED : serial monitor speed
#
# Global configuration.
ARD_REV ?= 100
ARD_HOME ?= /usr/local/arduino
ARD_BIN ?= /usr/bin
AVRDUDE ?= $(ARD_HOME)/hardware/tools/avrdude
AVRDUDE_CONF ?= $(ARD_HOME)/hardware/tools/avrdude.conf
MON_TERM ?= xterm
MON_SPEED ?= 57600
MON_CMD ?= picocom
PORT ?= $(HOME)/dev/arduino
BOARD ?= atmega328
### Nothing below here should require editing. ###
# Check for the required definitions.
ifndef BOARD
$(error $$(BOARD) not defined)
endif
ifndef PORT
$(error $$(PORT) not defined)
endif
# Version-specific settings
ARD_BOARDS = $(ARD_HOME)/hardware/arduino/boards.txt
ARD_SRC_DIR = $(ARD_HOME)/hardware/arduino/cores/arduino
ARD_MAIN = $(ARD_SRC_DIR)/main.cpp
# Standard macros.
SKETCH = $(notdir $(CURDIR))
BUILD_DIR = build
VPATH = $(LIB_DIRS)
# Macros derived from boards.txt
MCU := $(shell sed -n 's/$(BOARD)\.build\.mcu=\(.*\)/\1/p' < $(ARD_BOARDS))
F_CPU := $(shell sed -n 's/$(BOARD)\.build\.f_cpu=\(.*\)/\1/p' < $(ARD_BOARDS))
UPLOAD_SPEED := \
$(shell sed -n 's/$(BOARD)\.upload\.speed=\(.*\)/\1/p' < $(ARD_BOARDS))
PROGRAMMER := \
$(shell sed -n 's/$(BOARD)\.upload\.protocol=\(.*\)/\1/p' < $(ARD_BOARDS))
ARD_VAR := \
$(shell sed -n 's/$(BOARD)\.build\.variant=\(.*\)/\1/p' < $(ARD_BOARDS))
# More Version-specific settings
ARD_VAR_DIR = $(ARD_HOME)/hardware/arduino/variants/$(ARD_VAR)
# Build tools.
CC = $(ARD_BIN)/avr-gcc
CXX = $(ARD_BIN)/avr-g++
CXXFILT = $(ARD_BIN)/avr-c++filt
OBJCOPY = $(ARD_BIN)/avr-objcopy
OBJDUMP = $(ARD_BIN)/avr-objdump
AR = $(ARD_BIN)/avr-ar
SIZE = $(ARD_BIN)/avr-size
NM = $(ARD_BIN)/avr-nm
MKDIR = mkdir -p
RM = rm -rf
MV = mv -f
LN = ln -f
# Compiler flags.
INC_FLAGS = \
$(addprefix -I,$(INC_DIRS)) $(addprefix -I,$(LIB_DIRS)) -I$(ARD_SRC_DIR) -I$(ARD_VAR_DIR)
ARD_FLAGS = -mmcu=$(MCU) -DF_CPU=$(F_CPU) -DARDUINO=$(ARD_REV)
C_CXX_FLAGS = \
-Wall -Wextra -Wundef -Wno-unused-parameter \
-fdiagnostics-show-option -Wa,-adhlns=$(BUILD_DIR)/$*.lst
C_FLAGS = \
$(C_CXX_FLAGS) -std=gnu99 -Wstrict-prototypes -Wno-old-style-declaration
CXX_FLAGS = $(C_CXX_FLAGS)
### DEBUG Compilation ###
ifeq ($(DEBUG), 1)
ARD_FLAGS += -DDEBUG_PROJ
C_FLAGS += -g
CXX_FLAGS += -g
else
ARD_FLAGS += -DNDEBUG_PROJ
endif
# Optimiser flags.
# optimise for size, unsigned by default, pack data.
# separate sections, drop unused ones, shorten branches, jumps.
# don't inline, vectorise loops. no exceptions.
# no os preamble, use function calls in prologues.
# http://gcc.gnu.org/onlinedocs/gcc-4.3.5/gcc/
# http://www.tty1.net/blog/2008-04-29-avr-gcc-optimisations_en.html
OPT_FLAGS = \
-Os -funsigned-char -funsigned-bitfields -fpack-struct -fshort-enums \
-ffunction-sections -fdata-sections -Wl,--gc-sections,--relax \
-fno-inline-small-functions -fno-tree-scev-cprop -fno-exceptions \
-ffreestanding -mcall-prologues
# Build parameters.
IMAGE = $(BUILD_DIR)/$(SKETCH)
ARD_C_SRC = $(wildcard $(ARD_SRC_DIR)/*.c)
ARD_CXX_SRC = $(wildcard $(ARD_SRC_DIR)/*.cpp)
ARD_C_OBJ = $(patsubst %.c,%.o,$(notdir $(ARD_C_SRC)))
ARD_CXX_OBJ = $(patsubst %.cpp,%.o,$(notdir $(ARD_CXX_SRC)))
ARD_LIB = arduino
ARD_AR = $(BUILD_DIR)/lib$(ARD_LIB).a
ARD_AR_OBJ = $(ARD_AR)($(ARD_C_OBJ) $(ARD_CXX_OBJ))
ARD_LD_FLAG = -l$(ARD_LIB)
# Workaround for http://gcc.gnu.org/bugzilla/show_bug.cgi?id=34734
$(ARD_AR)(Tone.o) : CXX_FLAGS += -w
# Sketch libraries.
LIB_C_SRC = $(foreach ld,$(LIB_DIRS),$(wildcard $(ld)/*.c))
LIB_CXX_SRC = $(foreach ld,$(LIB_DIRS),$(wildcard $(ld)/*.cpp))
LIB_SRC = $(LIB_C_SRC) $(LIB_CXX_SRC)
ifneq "$(strip $(LIB_C_SRC) $(LIB_CXX_SRC))" ""
LIB_C_OBJ = $(patsubst %.c,%.o,$(notdir $(LIB_C_SRC)))
LIB_CXX_OBJ = $(patsubst %.cpp,%.o,$(notdir $(LIB_CXX_SRC)))
LIB_LIB = library
LIB_AR = $(BUILD_DIR)/lib$(LIB_LIB).a
LIB_AR_OBJ = $(LIB_AR)($(LIB_C_OBJ) $(LIB_CXX_OBJ))
LIB_LD_FLAG = -l$(LIB_LIB)
endif
# Sketch PDE source.
SKT_PDE_SRC = $(wildcard *.pde *.ino)
ifneq "$(strip $(SKT_PDE_SRC))" ""
SKT_PDE_OBJ = $(BUILD_DIR)/$(SKETCH)_pde.o
endif
# C and C++ source.
SKT_C_SRC = $(wildcard *.c)
SKT_CXX_SRC = $(wildcard *.cpp)
ifneq "$(strip $(SKT_C_SRC) $(SKT_CXX_SRC))" ""
SKT_C_OBJ = $(patsubst %.c,%.o,$(SKT_C_SRC))
SKT_CXX_OBJ = $(patsubst %.cpp,%.o,$(SKT_CXX_SRC))
SKT_LIB = sketch
SKT_AR = $(BUILD_DIR)/lib$(SKT_LIB).a
SKT_AR_OBJ = $(SKT_AR)/($(SKT_C_OBJ) $(SKT_CXX_OBJ))
SKT_LD_FLAG = -l$(SKT_LIB)
endif
# Definitions.
define run-cc
# $(CC) $(ARD_FLAGS) $(INC_FLAGS) -M -MT '$#($%)' -MF $#_$*.dep $<
$(CC) -c $(C_FLAGS) $(OPT_FLAGS) $(ARD_FLAGS) $(INC_FLAGS) \
$< -o $(BUILD_DIR)/$%
# $(AR) rc $# $(BUILD_DIR)/$%
# $(RM) $(BUILD_DIR)/$%
# $(CXXFILT) < $(BUILD_DIR)/$*.lst > $(BUILD_DIR)/$*.lst.tmp
# $(MV) $(BUILD_DIR)/$*.lst.tmp $(BUILD_DIR)/$*.lst
endef
define run-cxx
# $(CXX) $(ARD_FLAGS) $(INC_FLAGS) -M -MT '$#($%)' -MF $#_$*.dep $<
$(CXX) -c $(CXX_FLAGS) $(OPT_FLAGS) $(ARD_FLAGS) $(INC_FLAGS) \
$< -o $(BUILD_DIR)/$%
# $(AR) rc $# $(BUILD_DIR)/$%
# $(RM) $(BUILD_DIR)/$%
# $(CXXFILT) < $(BUILD_DIR)/$*.lst > $(BUILD_DIR)/$*.lst.tmp
# $(MV) $(BUILD_DIR)/$*.lst.tmp $(BUILD_DIR)/$*.lst
endef
# Rules.
.PHONY : all clean upload monitor upload_monitor
all : $(BUILD_DIR) $(IMAGE).hex
clean :
$(RM) $(BUILD_DIR)
$(BUILD_DIR) :
$(MKDIR) $#
$(SKT_PDE_OBJ) : $(SKT_PDE_SRC)
if [ $(ARD_REV) -ge 100 ]; then \
echo '#include "Arduino.h"' > $(BUILD_DIR)/$(SKETCH)_pde.cpp; \
else \
echo '#include "WProgram.h"' > $(BUILD_DIR)/$(SKETCH)_pde.cpp; \
fi
echo '#include "$(SKT_PDE_SRC)"' >> $(BUILD_DIR)/$(SKETCH)_pde.cpp
$(LN) $(SKT_PDE_SRC) $(BUILD_DIR)/$(SKT_PDE_SRC)
cd $(BUILD_DIR) && $(CXX) -c $(subst build/,,$(CXX_FLAGS)) \
$(OPT_FLAGS) $(ARD_FLAGS) -I.. \
$(patsubst -I..%,-I../..%,$(INC_FLAGS)) \
$(SKETCH)_pde.cpp -o $(#F)
(%.o) : $(ARD_SRC_DIR)/%.c
$(run-cc)
(%.o) : $(ARD_SRC_DIR)/%.cpp
$(run-cxx)
(%.o) : %.c
$(run-cc)
(%.o) : %.cpp
$(run-cxx)
$(BUILD_DIR)/%.d : %.c
$(run-cc-d)
$(BUILD_DIR)/%.d : %.cpp
$(run-cxx-d)
$(IMAGE).hex : $(ARD_AR_OBJ) $(LIB_AR_OBJ) $(SKT_AR_OBJ) $(SKT_PDE_OBJ)
$(CC) $(CXX_FLAGS) $(OPT_FLAGS) $(ARD_FLAGS) -L$(BUILD_DIR) \
$(SKT_PDE_OBJ) $(SKT_LD_FLAG) $(LIB_LD_FLAG) $(ARD_LD_FLAG) -lm \
-o $(IMAGE).elf
$(OBJCOPY) -O ihex -j .eeprom --set-section-flags=.eeprom=alloc,load \
--no-change-warnings --change-section-lma .eeprom=0 $(IMAGE).elf \
$(IMAGE).eep
$(OBJCOPY) -O ihex -R .eeprom $(IMAGE).elf $(IMAGE).hex
$(OBJDUMP) -h -S $(IMAGE).elf | $(CXXFILT) -t > $(IMAGE).lst
$(SIZE) $(IMAGE).hex
upload : all
- pkill -f '$(MON_CMD).*$(PORT)'
- sleep 1
- stty -F $(PORT) hupcl
- $(AVRDUDE) -V -C$(AVRDUDE_CONF) -p$(MCU) -c$(PROGRAMMER) -P$(PORT) \
-b$(UPLOAD_SPEED) -D -Uflash:w:$(IMAGE).hex:i
monitor :
LD_LIBRARY_PATH= LD_PRELOAD= \
$(MON_TERM) -title '$(BOARD) $(PORT)' \
-e '$(MON_CMD) -b $(MON_SPEED) $(PORT)' &
upload_monitor : upload monitor
-include $(wildcard $(BUILD_DIR)/*.dep))
# vim:ft=make
Use Example:
Given a dir-tree like follows:
Base_Dir
├── Library
│   ├── Base
│   │   ├── general STM8 sources and headers
│   ├── STM8S_Discovery
│   │   └── board specific sources and headers
│   └── User
│      └── optional user library sources and headers
├── Projects
│   ├── Examples (to be filled)
│   │   └── Basic_Project
│   │   ├── config.h
│   │   ├── example.ino
│   │   └── Makefile --> should detect dependencies in ./*.c and ./*.h
...
You could place Makefile.master within Projects, then assuming that:
You only need what is in Library/Base and Library/User for this project
You need to use LiquidCrystal Arduino Library in your project
then you would add the following Makefile into Basic Project:
# Your Arduino environment.
BASE_DIR = /path/to/Base_Dir # to edit
ARD_HOME = /usr/share/arduino # to edit, maybe
ARD_BIN = $(ARD_HOME)/hardware/tools/avr/bin
# Monitor Baudrate
MON_SPEED = 4800
# Board settings.
BOARD = uno
PORT = /dev/ttyACM0
PROGRAMMER = stk500v2
# Where to find header files and libraries.
INC_DIRS =
MY_LIB_DIRS= $(BASE_DIR)/Library/Base $(BASE_DIR)/Library/User
LIBS= LiquidCrystal
LIB_DIRS = $(addprefix $(ARD_HOME)/libraries/, $(LIBS)) $(MY_LIB_DIRS)
include ../../Makefile.master
Note that common.h should be automatically detected because it is located in ., and there should be no need to add the latter to INC_DIRS.
Final Note: last time I tested this configuration I was using version 1.0.5 of Arduino source code, and it was working flawlessly.

You can find a discussion, along with a sample implementation, of an automatic dependency generation method for GNU make here. You don't say you're using GNU make so I'm just assuming.
I don't know if that's sufficient for you or not; it wasn't clear from your statements about your requirements.

first of all thanks a lot to all for the fast and substantial support! I should've asked much earlier...
Now back to my (no longer an) issue. I now understand that I actually asked 2 different questions:
dependencies on headers -> solved by both Patrick's or MadScientist's proposal
detect what library is required by e.g. analysis of calls in main.o, or by the #includes in main.c...?
I understand the 2nd is much more difficult to achieve from within make...!? But Patrick's Makefile makes the manual configuration very convenient. So that's ok for me :-)
[add timeslip...] Ok, having pondered some more, would the following work / make sense?
call e.g. a gawk or python script from within Makefile to isolate all #included headers from the *.c and *.h in the project directory
for each included xyz.h check if a corresponding make_xyz exists in one of the Lib directories (would be part of the Lib)
if yes, include that makefile for compilation in the master Makefile
Does that somehow make sense...? Thanks again and have a nice day, wherever you are!
Regards, Georg Icking-Konert

thanks again for your support!
Following the above advice I wrote a small python script, which uses the gcc (or actually sdcc) dependency generator. The below script scans all project .c files for #included headers using gcc. The respective header files are then searched in the project and library folders. If a corresponding .c file exist (same path and name as header), it is added to the Makefile. This process is repeated until no more new headers are found.
The result is a Makefile which only builds modules which are #included modules in the project .c files - just as in the Arduino IDE. It may not be elegant but the job :-)
Lines 95-106 in the script are compiler and project specific, and have to be adapted accordingly. Have fun and thanks again!
#!/usr/bin/python
'''
automatically create a Makefile with dependencies from
all .c files in a starting directory
'''
# required modules
import sys
import os
import platform
import shlex
from subprocess import Popen, PIPE
# set OS specific
OS = platform.system()
if OS == 'Windows':
MAKE = 'mingw32-make.exe'
else:
MAKE = 'make'
##################
# helper functions
##################
#########
def getchar():
"""
python equivalent of getchar()
"""
ch = 0
if OS == 'Windows':
import msvcrt as m
ch = m.getch()
sys.stdio.flush()
sys.stderr.flush()
else:
import sys, tty, termios
fd = sys.stdin.fileno()
old_settings = termios.tcgetattr(fd)
tty.setraw(sys.stdin.fileno())
ch = sys.stdin.read(1)
return ch
# end getchar()
#########
def listFiles( start='.', pattern='.c' ):
"""
return set of matching files in project folder incl. subfolders
"""
result = set()
for root, dirs, files in os.walk(start):
for file in files:
if file.endswith(pattern):
#print(os.path.join(root, file))
result.add(os.path.join(root, file))
return result
# end listFiles()
#########
def listSubdirs( start='.' ):
"""
return set of subdirectories in given folder
"""
result = set()
for root, dirs, files in os.walk(start):
for dir in dirs:
#print(os.path.join(root, dir))
result.add(os.path.join(root, dir))
return result
# end listFiles()
#########
def get_exitcode_stdout_stderr(cmd):
"""
execute the external command and get its exitcode, stdout and stderr.
"""
args = shlex.split(cmd)
proc = Popen(args, stdout=PIPE, stderr=PIPE)
out, err = proc.communicate()
exitcode = proc.returncode
return exitcode, out, err
##################
# main program
##################
# set compile search paths
ROOT_DIR = '../../../'
TOOL_DIR = ROOT_DIR + 'Tools/'
LIB_ROOT = ROOT_DIR + 'Library/'
PRJ_ROOT = '.'
OBJDIR = 'output'
TARGET = 'main.ihx'
# set command for creating dependencies and set search paths
CC = 'sdcc '
CFLAGS = '-mstm8 --std-sdcc99 --std-c99 '
LFLAGS = '-mstm8 -lstm8 --out-fmt-ihx '
DEPEND = '-MM '
INCLUDE = '-I. '
for dir in listSubdirs(PRJ_ROOT):
INCLUDE += '-I' + dir + ' '
for dir in listSubdirs(LIB_ROOT):
INCLUDE += '-I' + dir + ' '
# get set of .c files in project folder incl. subdirectories
source_todo = listFiles(PRJ_ROOT,".c")
source_done = set()
header_done = set()
object_done = set()
# print message
sys.stdout.write('start Makefile creation ... ')
sys.stdout.flush()
# generate generic Makefile header
Makefile = open('Makefile', 'wb')
Makefile.write('OBJDIR = '+OBJDIR+'\n')
Makefile.write('TARGET = '+TARGET+'\n\n')
Makefile.write('.PHONY: clean all default objects\n\n')
Makefile.write('.PRECIOUS: $(TARGET)\n\n')
Makefile.write('default: $(OBJDIR) $(OBJDIR)/$(TARGET)\n\n')
Makefile.write('all: default\n\n')
Makefile.write('# create output folder\n')
Makefile.write('$(OBJDIR):\n')
Makefile.write(' mkdir -p $(OBJDIR)\n')
Makefile.write(' rm -fr -- -p\n\n')
# iteratively add project sources to Makefile
while (len(source_todo) > 0):
# get next pending source and mark as done
source = source_todo.pop()
source_done.add(source)
# convert Windows path to POSIX for Makefile
if OS == 'Windows':
source = source.replace('\\','/')
# use compiler generate dependency list
cmd = CC+DEPEND+CFLAGS+INCLUDE+source
#print cmd
exitcode, out, err = get_exitcode_stdout_stderr(cmd)
if (exitcode != 0):
print 'error: ' + err
getchar()
exit()
# append .c file with dependency and compile instruction to Makefile
Makefile.write('$(OBJDIR)/'+out)
#print(out)
Makefile.write('\t'+CC+CFLAGS+INCLUDE+'-c $< -o $#\n\n')
# extract file list including object[0], source[1] and headers[2..N]
out = out.replace(':', '')
out = out.replace('\\', '')
out = out.replace('\n', '')
out = out.split()
#print out
# for all files returned by compiler...
for next in out:
# append object files for linker
if next.endswith('.rel'):
object_done.add(next)
# if corresponding source to header exists, add to pending sources
if next.endswith('.h'):
if next not in header_done: # not yet in list
header_done.add(next) # add to treated headers
if (os.path.isfile(next[:-1]+'c')): # if corresponding .c exists, add to todo list
source_todo.add(next[:-1]+'c')
# link project object files
Makefile.write('$(OBJDIR)/$(TARGET): ')
for next in object_done:
Makefile.write('$(OBJDIR)/'+next+' ')
Makefile.write('\n')
Makefile.write('\t'+CC+LFLAGS)
for next in object_done:
Makefile.write('$(OBJDIR)/'+next+' ')
Makefile.write(' -o $#\n')
# close Makefile.dep
Makefile.close()
print('done\n')
sys.stdout.write('press any key to exit')
getchar()
exit()
# END OF MODULE

Related

Makefile handling auto-generated source and Makefile

I have a problem with a Makefile that's using an auto-generated Makefile to compile .o files from auto-generated .c and .s (assembler) source files.
The context is embedded programming for STM32 microcontrollers, where a Makefile and hardware initialization source code is generated from a hardware configuration file (.ioc file).
Suppose, we have the following directory structure:
.
├── archive
│   ├── a
│   │   └── a.c
│   ├── b
│   │   └── b.s
│   └── Makefile
├── Makefile
├── source
│   └── generate.sh
The files archive/a/a.c and archive/b/b.s may be empty here. My main ./Makefile looks like this:
.PHONY: default all clean generate objects
BUILD_DIR := build
SOURCE_DIR := source
OBJECTS := build/source/a/a.o build/source/b/b.o
default: all
all: objects
objects: $(OBJECTS)
$(BUILD_DIR)/%.o: %.c $(SOURCE_DIR)/Makefile
mkdir -p $(shell dirname $#)
$(MAKE) -C $(SOURCE_DIR) BUILD_DIR=../$(shell dirname $#) VPATH=../$(shell dirname $<) ../$#
$(BUILD_DIR)/%.o: %.s $(SOURCE_DIR)/Makefile
mkdir -p $(shell dirname $#)
$(MAKE) -C $(SOURCE_DIR) BUILD_DIR=../$(shell dirname $#) VPATH=../$(shell dirname $<) ../$#
$(SOURCE_DIR)/%.c :: generate
$(SOURCE_DIR)/%.s :: generate
$(SOURCE_DIR)/Makefile :: generate
generate: $(SOURCE_DIR)/generate.sh
cd $(SOURCE_DIR) ; bash generate.sh
clean:
$(RM) -r build $(SOURCE_DIR)/Makefile $(SOURCE_DIR)/a $(SOURCE_DIR)/b
The source/generate.sh looks like this:
cp -r ../archive/. ./
In reality, this process much more complicated and takes a few minutes.
Finally, the archive/Makefile looks like this:
BUILD_DIR := build
$(BUILD_DIR)/%.o: %.c
cp $^ $#
$(BUILD_DIR)/%.o: %.s
cp $^ $#
In reality, we use gcc instead of a simple cp to compile the C and ASM source files.
The auto-generated Makefile uses VPATH to find the required input file and is actually meant to compile everything in a flat build folder (which I don't want).
I have no control over the contents of the auto-generated Makefile! The source/generate.sh should be treated as a black-box that I have no influence on.
Therefore, the superordinate Makefile has to slightly abuse the auto-generated Makefile by overwriting the BUILD_DIR and VPATH accordingly.
Now, if I first run make generate and then, e.g. make build/source/a/a.o, everything works as planned:
First, the source directory is populated with the auto-generated stuff, and then build/source/a/a.o is "compiled" from source/a/a.c.
However, if I don't run make generate first, I get:
$ make build/source/a/a.o
make: *** No rule to make target 'build/source/a/a.o'. Stop.
My assumption was that the recipes would have been resolved in the following manner:
$(BUILD_DIR)/%.o: $(SOURCE_DIR)/%.c $(SOURCE_DIR)/Makefile
-> $(SOURCE_DIR)/%.c: generate
-> generate: $(SOURCE_DIR)/generate.sh
-> $(SOURCE_DIR)/Makefile: generate
-> generate: $(SOURCE_DIR)/generate.sh
After a bit of trial and error, I figured that I could somewhat fix this behaviour by defining the recipes as follows:
$(SOURCE_DIR)/%.c: generate
true
$(SOURCE_DIR)/%.s: generate
true
$(SOURCE_DIR)/Makefile: generate
true
This yields:
$ make build/source/a/a.o
cd source ; bash generate.sh
true
true
mkdir -p build/source/a
make -C source BUILD_DIR=../build/source/a VPATH=../source/a ../build/source/a/a.o
make[1]: Entering directory '/[...]/source'
cp ../source/a/a.c ../build/source/a/a.o
make[1]: Leaving directory '/[...]/source'
rm source/a/a.c
However, this leads to the generate step being run every time I run make build/source/a/a.o. Since the generate step is expensive in real life, this is not an option.
Also, source/a/a.c is treated as a temporary file that is meant to be deleted which I don't want.
How can I design my superordinate ./Makefile so that the generate step is being run automatically, but only if necessary?
I also need to be able to build non-auto-generated source files from the root directory.
A few more strange make behaviours:
Simply running make (i.e. make objects) with this Makefile yields
cd source ; bash generate.sh
true
true
mkdir -p build/source/a
make -C source BUILD_DIR=../build/source/a VPATH=../source/a ../build/source/a/a.o
make[1]: Entering directory '/[...]/source'
cp ../source/a/a.c ../build/source/a/a.o
make[1]: Leaving directory '/[...]/source'
true
mkdir -p build/source/b
make -C source BUILD_DIR=../build/source/b VPATH=../source/b ../build/source/b/b.o
make[1]: Entering directory '/[...]/source'
cp ../source/b/b.s ../build/source/b/b.o
make[1]: Leaving directory '/[...]/source'
rm source/a/a.c
Why does make remove source/a/a.c, but not source/b/b.c? For the record: I want neither of these files to be auto-removed.
An issue that I am sadly unable to reproduce with my example is that make basically outputs the following line at the start (adpated to the example):
make: Circular source/generate.sh.c <- generate dependency dropped.
Changing the recipe to the following
$(SOURCE_DIR)/%.c: generate
true
$(SOURCE_DIR)/%.s: generate
true
$(SOURCE_DIR)/Makefile: generate
yields:
$ make build/source/a/a.o
cd source ; bash generate.sh
true
cc -c -o source/Makefile.o source/Makefile.c
cc1: fatal error: source/Makefile.c: No such file or directory
compilation terminated.
make: *** [<builtin>: source/Makefile.o] Error 1
Why does make think that it needs to build a Makefile.o here?

Convert micropython makefile to cmakelist.txt

I am trying to convert micropython makefile to cmakelists.txt from https://docs.micropython.org/en/latest/develop/porting.html
# Include the core environment definitions; this will set $(TOP).
include ../../py/mkenv.mk
# Include py core make definitions.
include $(TOP)/py/py.mk
# Set CFLAGS and libraries.
CFLAGS = -I. -I$(BUILD) -I$(TOP)
LIBS = -lm
CROSS_COMPILE ?= arm-none-eabi-
# Define the required source files.
SRC_C = \
main.c \
mphalport.c \
lib/mp-readline/readline.c \
lib/utils/gchelper_generic.c \
lib/utils/pyexec.c \
lib/utils/stdout_helpers.c \
# Define the required object files.
OBJ = $(PY_CORE_O) $(addprefix $(BUILD)/, $(SRC_C:.c=.o))
# Define the top-level target, the main firmware.
all: $(BUILD)/firmware.elf
# Define how to build the firmware.
$(BUILD)/firmware.elf: $(OBJ)
$(ECHO) "LINK $#"
$(Q)$(CC) $(LDFLAGS) -o $# $^ $(LIBS)
$(Q)$(SIZE) $#
# Include remaining core make rules.
include $(TOP)/py/mkrules.mk
Current progress on cmakelists.txt
cmake_minimum_required(VERSION 3.17)
set(CMAKE_CXX_STANDARD 14)
set(MICROPY_TARGET firmware)
set(MICROPY_DIR "../..")
set(MICROPY_PORT_DIR ${CMAKE_CURRENT_SOURCE_DIR})
include(${MICROPY_DIR}/py/py.cmake)
project(${MICROPY_TARGET})
set(MICROPY_SOURCE_PORT
${MICROPY_PORT_DIR}/main.c
${MICROPY_PORT_DIR}/mphalport.c
${MICROPY_DIR}/lib/mp-readline/readline.c
${MICROPY_DIR}/lib/utils/gchelper_generic.c
${MICROPY_DIR}/lib/utils/pyexec.c
${MICROPY_DIR}/lib/utils/stdout_helpers.c
)
target_sources(app PRIVATE
${MICROPY_SOURCE_PY}
${MICROPY_SOURCE_PORT}
)
target_link_libraries(app PRIVATE ${MICROPY_TARGET})
include(${MICROPY_DIR}/py/mkrules.cmake)
I am getting error while running:
CMake error at line 33 :
can't specify sources for target "app" which is not built by this project.
I am taking help from this: https://github.com/micropython/micropython/blob/master/ports/zephyr/CMakeLists.txt , but I think I am missing some steps.

Qt static lib submodule build

I want to automate building thirdparty libraries and include them in my Qt project.
The .pro file of my Qt project looks like this:
QT -= gui
CONFIG += c++11 console
CONFIG -= app_bundle
QMAKE_EXTRA_TARGETS += stalib
PRE_TARGETDEPS += stalib
stalib.commands = make ../thirdparty/stalib/Makefile
LIBS += -L$${PWD}/../thirdparty/stalib/lib -lStalib
INCLUDEPATH += $${PWD}/../thirdparty/stalib/src
# The following define makes your compiler emit warnings if you use
# any feature of Qt which as been marked deprecated (the exact warnings
# depend on your compiler). Please consult the documentation of the
# deprecated API in order to know how to port your code away from it.
DEFINES += QT_DEPRECATED_WARNINGS
# You can also make your code fail to compile if you use deprecated APIs.
# In order to do so, uncomment the following line.
# You can also select to disable deprecated APIs only up to a certain version of Qt.
#DEFINES += QT_DISABLE_DEPRECATED_BEFORE=0x060000 # disables all the APIs deprecated before Qt 6.0.0
SOURCES += main.cpp
The Makefile of my thirdparty depedency submodule looks like this:
#-----------------------------------------------#
# Makefile for hello world #
#-----------------------------------------------#
# add preprocessor define
DEF_PARAMS = -DGREET=\"world\"
# Just to make sure
ifndef CROSS_COMPILE
$(error "cross-compilation environment not activated properly")
endif
# add debug symbols, DO NOT overwrite contents of FLAGS, append!
CFLAGS += -g $(DEF_PARAMS)
#Compilation only ignore warnings (ignore/-w, show all/-Wall).
CFLAGS += -c -w
SOURCEDIR=./src
LIBDIR=./lib
#-----------------------------------------------#
# Project Specific Settings #
#-----------------------------------------------#
# include directories relative to $SDKTARGETSYSROOT/usr/include (system HEADERS) or this Makefile (project headers).
INC_PARAMS = $(SOURCEDIR)
# project headers
HEADERS = $(SOURCEDIR)/math.h
# project sources
SOURCES = $(SOURCEDIR)/math.c
# Object files.
OBJECTS=$(SOURCES:%.c=%.c.o)
# Link libraries
# Libraries search directories relative to $SDKTARGETSYSROOT/usr/libs
# Library name without lib and .so e.g. libm.so -> -lm.
LINK_LIBS=
#Target name
TARGET_STATIC = $(LIBDIR)/libStalib.a
#-----------------------------------------------#
# Print Make Parameters #
#-----------------------------------------------#
print-%:
#echo "SOURCES=$(SOURCES)"
#echo "HEADERS=$(HEADERS)"
#echo "DEF_PARAMS=$(DEF_PARAMS)"
#echo "CFLAGS=$(CFLAGS)"
#echo "LDFLAGS=$(LDLAGS)"
#echo $* = $($*)
#-----------------------------------------------#
# Makefile Make Executable #
#-----------------------------------------------#
.SUFFIXES: .c
#Build rules begin.
all: $(TARGET_STATIC)
#Build rule for static library target.
$(TARGET_STATIC): $(OBJECTS)
$(AR) rc $# $(OBJECTS)
#Build rule for dynamic library target.
$(TARGET_SHARED): $(OBJECTS)
$(LD) $(LDFLAGS) $(OBJECTS) $(LINK_LIBS) -o $#
#Build rule for executeable target
$(TARGET): $(OBJECTS)
$(CC) $(LDFLAGS) $^ $(LINK_LIBS) -o $#
#Compilation rule for c files.
%.c.o: %.c $(HEADERS)
$(CC) $(CFLAGS) $(INC_PARAMS) $< -o $#
#Clean-up object files and target.
clean:
rm -f $(OBJECTS) $(TARGET) $(TARGET_STATIC) $(TARGET_SHARED)
However I get a linker error when building. It can not find the function defined in the math.h file:
#ifndef MATHH
#define MATHH
int addNums(int a, int b);
#endif
But the strange thing is that QtCreator is able to follow the reference to the header file.
For all of you who want to check the sources directly, or fiddle around with them:
https://github.com/faxe1008/myapp
https://github.com/faxe1008/stalib
Any help or suggestions on how to improve are appreciated.
If you want to build the library automatically, then you need to modify this line in your .pro:
stalib.commands = make -C../thirdparty/stalib CROSS_COMPILE=1
But that is not your question. You don't show your .cpp code, but I guess that you forgot to surround your #include like this:
extern "C" {
#include "math.h"
}
You can't include non system C headers in C++ sources without that. See: https://isocpp.org/wiki/faq/mixing-c-and-cpp

Error while executing cross-compiled program for openwrt

I'm trying to export a software to openwrt and i'm able to generate the ipk but the compiled file always gives me this error
-ash: scanReportProbe: not found
I've tried everything but i'm not able to make it work.
To be complete this is what i have done to prepare the environment:
git clone git://github.com/openwrt/openwrt.git
cd openwrt
./scripts/feeds update -a
./scripts/feeds install -a
make defconfig
make prereq
make menuconfig
[here i selected the package with the M and saved the configuration]
make tools/install
make toolchain/install
make package/scanReport_probe/compile
I have no idea of to fix this error so i need some hint of solution.
I don't think that the error is in the code but in casse is needed this is my program:
file structure
scanReport_probe
├── LICENSE
├── Makefile
├── README.md
└── src
├── data
│ └── THIS IS FOR DATA FILES.txt
├── runScript
│ └── scanReportProbe
├── scanReport_probe
│ ├── a.c
│ └── makefile
└── scanReport_probe.conf
makefile for openwrt
include $(TOPDIR)/rules.mk
# name of package
# version number of the sources you're using
# how many times you've released a package based on the above version number
PKG_NAME:=scanReport_probe
PKG_VERSION:=1.0
PKG_RELEASE:=1
PKG_LICENSE:=MIT
# sources will be unpacked into this directory (you shouldn't need to change this)
PKG_BUILD_DIR:=$(BUILD_DIR)/$(PKG_NAME)-$(PKG_VERSION)
#needed
include $(INCLUDE_DIR)/package.mk
# Metadata; information about what the package is for the ipkg listings
# to keep things simple, a number of fields have been left at their defaults
# and are not shown here.
define Package/scanReport_probe
SECTION:=utils
CATEGORY:=Utilities
TITLE:=scanReport_probe
URL:=https://github.com/luigiDB/scan-report_probe
TITLE:=Scan in monitor mode probe over wifi channel with periodic report to a server through http post.
MAINTAINER:=Please refer to github repository page
endef
define Package/scanReport_probe/description
This is the best tool ever.
Scan probe over wifi and at regualar interval send results via http post.
endef
define Build/Prepare
mkdir -p $(PKG_BUILD_DIR)
$(CP) ./src/* $(PKG_BUILD_DIR)/
endef
CONFIGURE_VARS+= \
CC="$(TOOLCHAIN_DIR)/bin/$(TARGET_CC)"
define Build/Compile
$(MAKE) -C $(PKG_BUILD_DIR)/scanReport_probe $(TARGET_CONFIGURE_OPTS)
endef
# We'll use the OpenWrt defaults to configure and compile the package. Otherwise we'd need to define
# Build/Configure - commands to invoke a configure (or similar) script
# Build/Compile - commands used to run make or otherwise build the source
define Package/scanReport_probe/install
# Now that we have the source compiled (magic huh?) we need to copy files out of the source
# directory and into our ipkg file. These are shell commands, so make sure you start the lines
# with a TAB. $(1) here represents the root filesystem on the router.
# INSTALL_DIR, INSTALL_BIN, INSTALL_DATA are used for creating a directory, copying an executable,
# or a data file. +x is set on the target file for INSTALL_BIN, independent of it's mode on the host.
# make a directory for the config
$(INSTALL_DIR) $(1)/etc/scanReport_probe/
# copy the config
$(INSTALL_CONF) $(PKG_BUILD_DIR)/scanReport_probe.conf $(1)/etc/scanReport_probe
# make a directory for some random data files required by scanReport_probe
$(INSTALL_DIR) $(1)/usr/share/scanReport_probe
# copy the data files
$(INSTALL_DATA) $(PKG_BUILD_DIR)/data/* $(1)/usr/share/scanReport_probe
#make directory bin
$(INSTALL_DIR) $(1)/bin
# copy the binary
$(INSTALL_BIN) $(PKG_BUILD_DIR)/scanReport_probe/scanReportProbe $(1)/bin
#make the directory init.d in case isn't present
$(INSTALL_DIR) $(1)/etc/init.d
#copy script to init.d
$(INSTALL_BIN) $(PKG_BUILD_DIR)/runScript/scanReportProbe $(1)/etc/init.d
endef
#runned post installation call the enable on the service
define Package/scanReport_probe/postinst
#!/bin/sh
# check if we are on real system
if [ -z "$${IPKG_INSTROOT}" ]; then
echo "Enabling rc.d symlink for scanReportProbe"
/etc/init.d/scanReportProbe enable
fi
exit 0
endef
#runned pre uninstallation call the disable on the service
define Package/scanReport_probe/prerm
#!/bin/sh
# check if we are on real system
if [ -z "$${IPKG_INSTROOT}" ]; then
echo "Removing rc.d symlink for scanReportProbe"
/etc/init.d/scanReportProbe disable
fi
exit 0
endef
$(eval $(call BuildPackage,scanReport_probe))
a.c
#include <stdio.h>
int main(void)
{
printf("Hello World\n");
return 0;
}
makefile for c program
PROFILE = -O2 -s
CFLAGS = $(PROFILE)
LDFLAGS =
all: main
# build it
main: a.o
$(CC) $(LDFLAGS) a.o -o scanReportProbe
a.o: a.c
$(CC) $(CFLAGS) -c a.c
# clean it
clean:
rm *.o a
For -ash, IIRC, ash is a [boot] shell. It is saying that it can't find a particular command.
So, it may be one of:
(1) In your rc script section, you're doing /etc/init.d/scanReportProbe enable but the package you're building uses scanReport_probe. So, you may need to add or remove an _ here or there and/or some other renames so things match up.
(2) If the file structure you gave is the final one, I'm not sure the script is in the right directory.
(3) Are the permissions for the script correct (i.e. has execute)?

Geting source code structure in Makefile

I'm working on a C project, and I decided to put the source code and its objects in different directories. The root directory has something like that:
SmartC ▶ tree -L 1
.
├── built
├── doc
├── Makefile
├── README.md
├── src
├── tests
└── trash
So, inside both src and built directories, I put two others Makefiles to do the compile and link jobs.
The src directory (where I put the source code) has the following structure:
src
├── graph.c
├── graph.h
├── list.c
├── list.h
├── main.c
├── Makefile
├── node.c
├── node.h
├── tree.c
├── tree.h
└── types
├── complex.c
├── complex.h
├── matrix.c
└── matrix.h
and the built has the same structure, but it is intended to store all objects made by compilation.
My question is about my src/Makefile:
BINDIR = ../built/src
CC = gcc
CFLAGS = -g -Wall -O3
OBJECTS = \
$(BINDIR)/main.o \
$(BINDIR)/node.o \
$(BINDIR)/list.o \
$(BINDIR)/graph.o \
$(BINDIR)/tree.o \
$(BINDIR)/types/complex.o \
$(BINDIR)/types/matrix.o \
compile: $(OBJECTS)
$(BINDIR)/%.o: %.c
$(CC) -c $< -o $# $(CFLAGS)
This Makefile creates all the objects of the source code, inside src directory, and move them to built/src. But, every time I create a new source code file (*.c), I have to put the name of its object in this makefile, so it can be compiled. I'd like to do an automatic search, inside the src directory, and fill the "OBJECTS" variable with this search.
Is anyone has some idea of how to accomplish this? I mean, automatic search for source code inside an specific directory?
I even accept any other strategy rather than what I'm making.
=========== Answer ===============
I got the tip (in comments) about wildcards. So I did. Here is the solution I found.
src/Makefile
BINDIR = ../built/src
CC = gcc
CFLAGS = -g -Wall -O3
OBJECTS := $(patsubst %.c,$(BINDIR)/%.o,$(wildcard *.c */*.c))
compile: $(OBJECTS)
$(BINDIR)/%.o: %.c
$(CC) -c $< -o $# $(CFLAGS)
EDIT [Solved]
I like to do the following.
Create Variables to Each Directory of the Project
SRCDIR = src
OBJDIR = obj
LIBDIR = lib
DOCDIR = doc
HDRDIR = include
CFLAGS = -g -Wall -O3
Get Only the Internal Structure of SRCDIR Recursively
STRUCTURE := $(shell find $(SRCDIR) -type d)
Get All Files inside the STRUCTURE Variable
CODEFILES := $(addsuffix /*,$(STRUCTURE))
CODEFILES := $(wildcard $(CODEFILES))
Filter Out Only Specific Files
# Filter Only Specific Files
SRCFILES := $(filter %.c,$(CODEFILES))
HDRFILES := $(filter %.h,$(CODEFILES))
OBJFILES := $(subst $(SRCDIR),$(OBJDIR),$(SRCFILES:%.c=%.o))
# Filter Out Function main for Libraries
LIBDEPS := $(filter-out $(OBJDIR)/main.o,$(OBJFILES))
Now it is Time to create the Rules
compile: $(OBJFILES)
$(OBJDIR)/%.o: $(addprefix $(SRCDIR)/,%.c %.h)
$(CC) -c $< -o $# $(CFLAGS)
With this approach, you can see that I'm using the STRUCTURE variable only to get the files inside the SRCDIR directory, but it can be used for others purposes as well, like mirror the SRCDIR inside OBJDIR once STRUCTURE stores only the internal sub-directories. It is quite useful after clean operations like:
clean:
-rm -r $(OBJDIR)/*
NOTE: The compile rule only works well if for each *.c there is the corresponding *.h file (with the same base name, I mean).

Resources