I'm currently working on a C project with one main executable and one executable for each unit test. In the SConstruct file I specify the dependencies for each executable, something like
env.Program(['Main.c', 'Foo.c', 'Bar.c', 'Baz.c', ...])
env.Program(['FooTest.c', 'Foo.c', 'Baz.c', ...])
env.Program(['BarTest.c', 'Bar.c', 'Baz.c', ...])
...
This, however, is error prone and inelegant since the dependencies could just as well be tracked by the build tool, in this case SCons. How can I improve my build script?
What you are asking for is some sort of tool that
1) Looks at the headers you include
2) Determines from the headers which source files need building
3) Rinse and repeat for all the source files you've just added
Once it's done that it'll have to look over the tree it has generated and try and squish some of that into sensible libraries, assuming you haven't done that already (and looking at the tone of both the questions, that exercise seems to have been viewed as academic, rather than a standard part of good software development).
There might be some mileage in a tool that says "You've included header A/B.h, so you'll need libA in your link line" but even that is going to have plenty of gotchas depending on how different people build and link their libraries.
But what you've asked is asking how to define a build script that writes a build script. It's something you should be doing for yourself.
Related
I want to know please if there is a method to add sources files automatically from a specific directory without writing them in the Makefile.am . So I need an option if this exist. Thanks for help.
I want to know please if there is a method to add sources files automatically from a specific directory without writing them in the Makefile.am .
Some make implementations, such as GNU's, have features that address this objective, but there is no portable mechanism for it, and the Autotools are all about portability. It would therefore be poor Autotools style to rely on such a mechanism.
Even if you were satisfied to ignore portability implications, there would still be a limited scope for this, because Makefile.am associates source files with the target(s) to which they contribute. You would need there to be a direct link between the source directory in question and the target, or possibly between more specific filename patterns and targets. Sometimes such associations exist, but other times they don't.
Portability questions aside, I am not much of a fan of this sort of thing in general, as it lays a trap. If you engage such a mechanism then it is hard to add a source file without breaking the build, for example. The one-time cost of adding all the source names when autotooling an existing project is not that bad -- I've done it for several large projects -- and the cost to maintain your Makefile.am when you add sources is trivial.
But if you nevertheless want something along these lines then I would suggest engaging a source generator. It might look like this:
Makefile.am
bin_PROGRAMS = myprog
include myprog_sources.am
generate_myprog_sources.sh
#!/bin/bash
myprog_sources=(src/myprog/*.c)
echo "myprog_SOURCES = ${myprog_sources[*]}" > myprog_sources.am
That uses a separate shell script to generate Automake code for a _SOURCES definition naming all the .c files in the specified directory (that is, src/myprog). That's written to its own file, myprog_sources.am, for simplicity and safety, and the Makefile.am file uses an Automake include directive to incorporate it. When you want to adjust the build for a different complement of source files, you run the script. You can also make temporary changes to the source list by manually updating it without running the script.
This is not quite the level of hands-free automation that you asked for, but it's still pretty automatic, and it's better suited to the tools you are using.
I want to require a c library which was build by with the autotools.
To be honest I have little to no idea how they work :/
(The library which I want to require is "https://github.com/p4lang/PI")
I have executed the ./configure etc. scripts and successfully installed it.
When I search my usr I find the library under /usr/local/lib/libpi.a
and analogously the header files under /usr/local/include/PI.
I build my project with cmake and would like to have a cross platform solution with it.
However I would be satisfied to use the pkg-config command.
Does anybody know what is the "correct" / "recommended" way to get cflags,
or at least a variant in which I do not have to hard code the paths?
The involvement of the Autotools ends at the point where the built artifacts are installed on the system. Using those does not go through the Autotools.* This applies just as much when the installed artifacts are libraries and headers as when they are executables. There's nothing special or different about using Autotools-built programs or libraries.
I build my project with cmake and would like to have a cross platform
solution with it. However I would be satisfied to use the pkg-config
command.
Just like projects served by any other build system, Autotools projects can build and install pkg-config configuration files, or CMake macros, or whatever other bits and pieces they might think appropriate to assist users, but this is project-specific. The Autotools do not create such additional pieces of their own accord, but some Autotools-based projects do add them. And some don't, just like some CMake projects don't, and some projects with hand-rolled build systems don't, etc..
Does anybody know what is the "correct" / "recommended" way to get cflags, or at least a variant in which I do not have to hard code the paths?
Note that typically, for a library whose name you know, the only flags you might need are those specifying the location of the library headers and / or one specifying the location of the libraries themselves. Even these are unnecessary if the relevant pieces are installed in places that the compiler looks by default. Also these are generally not considered CFLAGS, per se. Terminology varies a bit, but the former is a preprocessor flag, and the latter is a link flag.
Since you're using CMake, you could consider writing CMake code to search likely directories for the wanted libraries and headers, and to set the results in suitable variables for other code to use. That's more of an Autotools-style approach, though. Alternatively, you could define a user-set variable by which the wanted location(s) can be specified to CMake. This assumes that the third-party project is not already providing something useful for the purpose. Or, licensing permitting, you could package the third-party library together with your own, so that you are in control of where it gets installed.
In the general case, however, this is simply something that people have to deal with themselves when they build software. Make life easier for them by providing good documentation of what your project's dependencies are, and of how to inform the build system of their locations, and make useful provisions for feeding that information into the build system.
*An exception could be asserted for use of libtool archives, which an Autotools project might install alongside regular libraries -- if one wanted to use those, they would directly or indirectly go through libtool. But in practice, that's only going to happen in another Autotools project.
I have been reading up on make and looking at the Makefiles for popular C projects on GitHub to cement my understanding.
One thing I am struggling to understand is why none of the examples I've looked at (e.g. lz4, linux and FFmpeg) seem to account for header file dependencies.
For my own project, I have header files that contain:
Numeric and string constants
Macros
Short, inline functions
It would seem essential, therefore, to take any changes to these into account when determining whether to recompile.
I have discovered that gcc can automatically generate Makefile fragments from dependencies as in this SO answer but I haven't seen this used in any of the projects I've looked at.
Can you help me understand why these projects apparently ignore header file dependencies?
I'll attempt to answer.
The source distros of some projects include a configure script which creates a makefile from a template/whatever.
So the end user which needs to recompile the package for his/her target just has to do:
$ configure --try-options-until-it-works
$ make
Things can go wrong during the configure phase, but this has nothing to do with the makefile itself. User has to download stuff, adjust paths or configure switches and run again until makefile is successfully generated.
But once the makefile is generated, things should go pretty smooth from there for the user which only needs to build the product once to be able to use it.
A few portion of users will need to change some source code. In that case, they'll have to clean everything, because the makefile provided isn't the way the actual developpers manage their builds. They may use other systems (code::blocks, Ant, gprbuild...) , and just provide the makefile to automate production from scratch and avoid to depend on a complex production system. make is fairly standard even on Windows/MinGW.
Note that there are some filesystems which provide build audit (Clearcase) where the dependencies are automatically managed (clearmake).
If you see the makefile as a batch script to build all the sources, you don't need to bother adding a dependency system using
a template makefile
a gcc -MM command to append dependencies to it (which takes time)
Note that you can build it yourself with some extra work (adding a depend target to your makefile)
Basically, I want to seperate some common functionality from existing projects into a seperate library project, but also allow a project to remain cross-platform when I include this library.
I should clarify that when I say "cross-platform" I'm primarily concerned with compiling for multiple CPU architectures (x86/x86_64/ARM).
I have a few useful functions which I use across many of my software projects. So I decided that it was bad practice to keep copying these source code files between projects, and that I should create a seperate library project from them.
I decided that a static library would suit my needs better than a shared library. However, it occurred to me that the static library would be plaform dependent, and by including it with my projects that would cause these projects to also be platform dependent. This is clearly a disadvantage over including the source code itself.
Two possible solutions occur to me:
Include a static library compiled for each platform.
Continue to include the source code.
I do have reservations about both of the above options. Option 1 seems overly complex/wasteful. Option 2 seems like bad practice, as it's possible for the "library" to be modified per project and become out-of-sync; especially if the library source code is stored in the same directory as all the other project source code.
I'd be really grateful for any suggestions on how to overcome this problem, or information on how anyone else has previously overcome this problem?
You could adopt the standard approach of open source project (even if your project is not open source). There would be one central point where one can obtain the source code, presumably under revision control (subversion, git...). Anyone who wishes to use the library should check out the source code, compile it (a Makefile or something similar should be included), and then they are all set. If someone needs to change something in the library, they do so, test their changes, and send you a patch so that you can apply the change to the project (or not, depending on your opinion on the patch).
Apologies if I explain this badly or am asking something bleeding obvious but I'm new to the Linux kernel and kinda in at the deep end...
We have an embedded-linux system which arrives with a (very badly documented) SDK containing hundreds of folders of stuff, most folders containing a rules.make, make, make.config or some variation of... and the root folder containing a "master" makefile & rules.make which mean that you can, from the root folder, type "make sysall" and it builds the entire package.
So far so good, but trying to debug it is a bit of an issue as the documentation will say something like:
"To get the kernel to output debug messages, just define #outputdebugmessagesplz"
OK, but some of these things are defined in the "master" make/rules file, some of these are defined in the child make/rules/config files, some are in .h files... and of course it's far nicer to turn these things on/off from the "top" make.config rather than modifying individual .h files and then having to remember to turn them off again.
So I thought it would be a useful thing to recursively build a tree, starting from the master "make" file and following everything it does, everything that gets defined or re-defined, etc... but there doesn't seem to be a simple way of doing that?
I assume I am missing a "make" option here that spits this info out, or a usage of the makefile/config that will just work?
Your situation is not uncommon. When developing for embedded systems, you might encounter many custom systems that solve a problem in a specific way. As people already commented on your question, there's no easy way to generate a dependency graph for your makefile structure/framework. But there are some things you can try, and I'll try to base my suggestions based on your situation. Since you've said:
Im new to the Linux kernel and kinda in at the deep end...
and
We have an embedded-linux system which arrives with a (very badly
documented) SDK containing hundreds of folders of stuff
You could try the following things:
If your SDK is provided by a third-party vendor, try contacting them and get some support.
SDK's usually provide an abstraction to work with several components without a deep understanding of how each one of them really works. Try to pinpoint your problem, like if you want to customize only the kernel configuration, you could find the linux kernel folder on your SDK (assuming your SDK is composed of a set of folders with things like libraries, source code of applications and stuff, one of them might be the kernel one) and run make menuconfig. This will open a ncurses-based configuration GUI that you can navigate and choose kernel options.
As people already pointed out, you can try to run make -n and check the output. You could also try to run make -p | less and inspect the output, but I don't recommend this since it will only print the data base (rules and variable values) that results from reading the makefiles. You would have to parse this output to find out what you want in it.
Basically, you should try to pinpoint what you want to customize and see how this interacts with your SDK. If it's the kernel, then working only with it will give you a starting point. The linux kernel has its own makefile-build system, named kbuild. You can find more information about it at the kernel's Documentation folder.
Besides that, trying to understand how makefiles work will help you if you have a complex makefile structure controlling several components. The following are good resources to learn about makefiles:
GNU Make official documentation
O'Reilly's Open Book "Managing Projects with GNU Make"
Also, before trying to build your own tool, you can check if there's an open source project that does what you want. A quick search on google gave me this:
makegrapher
Also, check this question and this one. You might find useful information from people that had the same problems as you did.
Hope it helps!