What will (getDirectoryFiles "" (dir </> "//*.c")) actually match? - shake-build-system

I was experiencing unnecessary rebuilds while using this matching pattern:
getDirectoryFiles "" $ "foo" </> "//*.c"
I'm on Linux so from my understanding the second argument will evaluate to
"foo///*.c"
and I presume that it would be incorrect to replace this with
"foo" </> "/*.c"
as that would not work on windows? On Linux however this works as I expect it to.
From testing I concluded that the pattern matched files outside of "foo" which makes no sense in my world. As an example the pattern above would match both testA.c and testB.c in a directory structure like the one below.
foo/testA.c
bar/testB.c
I see now that there is a <//> operator, so maybe I should be using "foo" <//> "*.c" instead but still. This was quite nasty to find and what does "foo///*.c" actually do?

The problem is that the </> operator is defined so that:
"foo" </> "//*.c" == "//*.c"
That means that using </> to create a // value is almost never the right thing to do. <//> solves that, but as you note, it's not predictable and quite nasty.
The solution is using foo/**/*.c as an alternative pattern to foo//*.c. The advantage of the first form is that it is perfectly predictable and works well with </>, eliminating the need for <//>. Since Shake 0.15.6 that has been a permissible alternative, and in Shake 0.16 it will probably be recommended over //.
As to your other questions: 1) for Windows you can use either / or \ interchangeably in pattern values, so writing foo//*.c literally is just fine. 2) foo///*.c is equivalent to foo//*.c, and there are extensive test suites for such forms in Shake.

Related

Makefile filter-out just... isn't working?

Okay, I'm going to include some backstory about what I'm trying to accomplish, feel free to skip to the end for an example that demonstrates what I'm talking about.
Right at the moment, I'm trying to get a test framework set up around my code, with as minimal of an impact on the actual code base as possible (I'm considering a couple other frameworks right now, and want to make switching as painless as possible). The approach that I'm using currently is to link all of my project's object files together with my testing code so I don't have to #include the source files or use some other hack.
This approach creates a problem with the main function for obvious reasons; my current plan is to simply not link the object file from my project that defines main. This would prevent me from testing it, but it's really simple and any issues will be really obvious, so I'm okay with that. (I'd say this approach beats the alternative of screwing with the symbol table in main.o, which is the only alternative I can think of at the moment.)
The issue so far is that I can't seem to get make to actually filter out main.o from the list of object files I'm linking together, see below.
The promised demo of what I'm talking about:
MAIN_PROG_OBJ_DIR ::= build/test/release/main-prog-obj
# In the actual Makefile, this is generated as usual
MAIN_PROG_OBJS ::= build/test/release/main-prog-obj/state.o build/test/release/main-prog-obj/main.o build/test/release/main-prog-obj/util.o build/test/release/main-prog-obj/process_args.o
MAIN_O ::= $(MAIN_PROG_OBJ_DIR)"/main.o"
OBJS_FOR_BUILD ::= $(filter-out $(MAIN_O),$(MAIN_PROG_OBJS))
.PHONY: test
test:
#echo should be removed [$(MAIN_O)]
#echo OBJS_FOR_BUILD [$(OBJS_FOR_BUILD)]
on my system, running this with make -f testmk test yields:
should be removed [build/test/release/main-prog-obj/main.o]
OBJS_FOR_BUILD [build/test/release/main-prog-obj/state.o build/test/release/main-prog-obj/main.o build/test/release/main-prog-obj/util.o build/test/release/main-prog-obj/process_args.o]
Clearly I'm misunderstanding what $(filter-out does/what its syntax is, so how do I accomplish what I'm trying to do?
Thanks!
It was a quoting issue, make was interpreting "main.o" literally and therefore wouldn't filter out main.o from the list. The confusion came from the fact that, when I echo'd "main.o", the shell parsed the quotes and just passed main.o to the echo built-in.

how to rescan m4 data for recursive macro inplace substitution

I have this very simple code.
define(`S',`some')
define(`T',`thing')
define(`D',`doing')
define(`something',`st_todo')
define(`st_tododoing',`done!')
S`'T
OR
S()T()
the actual end result is
something
OR
something
but expected recursive substitution result as
st_todo
how I can rescan the code to the input again ?
maybe the "indir", But its A crooked nasty solution:
indir(S()`'T()`'D())
the result is:
--NOTHING--
maybe the so called command "divert", can make the output, recycled to input ?
of course the reviews says it is "inplace macro substitution",
BUT the result are different.
of course we can use "C" language primitive macro solution. EG:
define(`concat',`$1$2$2$4$5')
BUT this "concat" "solution", will increase the code NESTING complexity, in a large code reconstruction. EG:
concat(concat(S1,T1),concat(S2,T2,more1,more2,...))
WHAT if we have "concat" of 10 words OR more OR with conditionals "ifelse" ?.
M4 concept is beyond the old "C" preprocessor!
the real solution must come from the macro "inside core system"
any idea ?
M4 does rescan. However, the empty string `' ends the preceding token, and thus prevents some and thing from being recognised as a single token.
Instead, concatenate the macro expansions using another macro:
define(`concat',`$1$2')
define(`S',`some')
define(`T',`thing')
define(`something',`st_todo')
concat(S,T)
$ m4<<"EOF"
> define(`concat',`$1$2')dnl
> define(`S',`some')dnl
> define(`T',`thing')dnl
> define(`something',`st_todo')dnl
> concat(S,T)
> EOF
st_todo

Is it possible for Shake to change a source file?

When running tools such as formatters and linting tools with "auto-correction" options, it can be that the input and output for a Rule are the same file; for example:
"//*.hs" %> \out ->
cmd_ "ormolu" "-m" "inplace" out
-- OR
batch 10 ("//*.hs" %>)
( \out -> do
cmd_ "ormolu" "-m" "inplace" out
pure out
)
(cmd_ "hlint")
This seems to work "correctly" (the rule is re-run if the source file is needed and has changed), but we're unsure if this is a happy coincidence or shake working as designed - especially when we start thinking about cached results from shakeShare or in the future Cloud Shake. Is this the best way to handle this type of rule, or is there something better?
There is no principled way to generate a rule that replaces a source file in Shake. Given a source code formatter, anything else isn't very usfeul. Shake makes the assumption that inputs don't change while the compilation is ongoing. It's likely that passing --lint will lead to a lint error and that it would be incompatible with Cloud Shake. The official advice would be to make such changes in a separate non-Shake pass before you call shake.
However, if it works for you, and is useful, I wouldn't overly worry. The pattern has tests in Shake, it's something plenty of people do. You can turn off Cloud caching on a per file basis with historyDisable.

What is the correct usage of have_func when the test function needs arguments?

I have an extconf.rb with the following lines:
have_header("cblas.h") # checking for cblas.h... yes
have_func("cblas_dgemm", ["cblas.h"]) # checking for cblas_dgemm() in cblas.h... no
create_header("nmatrix_config.h") # creating nmatrix_config.h
So, cblas_dgemm is definitely in cblas.h. When I look at mkmf.log, I see that this check actually looks for two things:
a _cblas_dgemm symbol somewhere (?)
a callable cblas_dgemm in cblas.h.
Both tests are failing. I assume the former is failing because I need a dir_config line for cblas, and maybe a have_library('cblas').
But I can't figure out how to make the latter test pass (see line 24 of the gist). Is it possible to pass a block to have_func so it actually calls it with reasonable arguments? Or is there some other way to run this test? Or do I have to have the dir_config stuff setup properly?
Here's line 24, by the way:
conftest.c:7:1: error: too few arguments to function ‘cblas_dgemm’
And yes, of course, cblas_dgemm needs many arguments -- some of them matrices.
It's frustrating how little documentation there is on any of these mkmf functions.
Unfortunately it looks like have_func is pretty poorly documented, but upon some digging I found something that might help:
[25] pry(main)> have_func("clapack_dgetrf", "/usr/local/atlas/include/clapack.h")
checking for clapack_dgetrf() in /usr/local/atlas/include/clapack.h... no
=> false
[26] pry(main)> have_func("int clapack_dgetrf", "/usr/local/atlas/include/clapack.h")
checking for int clapack_dgetrf() in /usr/local/atlas/include/clapack.h... yes
=> true
So, essentially it looks like you need to include at least the return type to get have_func to work properly. Can you verify that this works on your machine?
In mkmf.rb of ruby 1.9.3p392, The latter test is executed only if the former fails. When the former passes, have_func successes. So you don't have to make the latter pass. Refer to try_func in mkmf.rb for more details.
Just for information, in mkmf.rb on 2013-04-13 11:00:25, it seems you can give arguments like this: have_func("some_func(some_arg, another_arg)", ["foo.h"]).

Portable access to sysconfdir via config.h

I'd like my application to have portable access to the configuration files installed during make install (dist_sysconf_DATA). Is it possible to access $(sysconfdir) via config.h?
It is, but you should not do this according to official voices (as in, I am not gonna search the manual for it now) so as to continue supporting overriding it for specific objects to be built.
make CPPFLAGS="-USYSCONFDIR -DSYSCONFDIR=/blah" thisoneobject.o
Hence, what one is supposed to do:
AM_CPPFLAGS = -DSYSCONFDIR=\"${sysconfdir}\"
If you're using autoheader, adding this to your configure.ac will output a SYSCONFDIR macro in your config.h, and it will be defined with the value $(sysconfdir) or ${prefix}/etc.
if test "x$sysconfdir" = 'x${prefix}/etc'; then
if test "x$prefix" = 'xNONE'; then
sysconfdir=$ac_default_prefix/etc
else
sysconfdir=$prefix/etc
fi
fi
AC_DEFINE_UNQUOTED([SYSCONFDIR], ["$sysconfdir"], [location of system configuration directory])
But I would strongly recommend against doing that, and instead, stick with using the -DSYSCONFDIR flag. It's less code and therefore less prone to something going wrong. Using a condition in configure.ac such I mentioned may not be portable or take into account every case that might be encountered. Using -DSYSCONFDIR is the best option. Sometimes appearance just doesn't matter.
What I believe is most commonly done (and this is what I do)
Add the following in your Makefile.am
AM_CPPFLAGS = -DSYSCONFIR='"$(sysconfdir)"'
And now you can access SYSCONFDIR in source

Resources