Is there a built-in way to pass command-line arguments to a "shakefile"? I'd like to pass --env production|development|staging and then use it within my rules to (slightly) alter the build-steps for each environment.
There are two halves to this problem - first getting the flags into Shake, and secondly, using them to influence the behaviour.
You can get arguments into Shake using any Haskell command line parser, but Shake ships with support for one built it, which can often be easier:
data Flags = Production | Dev | Staging deriving Eq
flags = [Option "" ["production"] (NoArg $ Right Production) "Build production."
,Option "" ["dev"] (NoArg $ Right Dev) "Build dev."
,Option "" ["staging"] (NoArg $ Right Staging) "Build staging."
]
main = shakeArgsWith shakeOptionsn flags $ \flags targets -> do
want targets
... do whatever you want with the flags ...
return rules
For using the flags to influence, you might want to:
Completely isolate build outputs from each flag, in which case changing the directory and setting shakeFiles differently in each case makes each one fully distinct.
Use the flags to change the output paths, so you always have rules dev/main.js and prod/main.js, and then you consult the flags when doing want to pick up the right rules.
Put the flags into an Oracle and have them as tracked settings, so when you flip from prod to dev some things rebuild.
If the builds are 80%+ distinct I'd go for 1. If you change flags very rarely 3 can work. Otherwise, I tend to opt for 2. But they all work, so picking the simplest to start with is also not unreasonable.
Related
Most Linux apps are compiled with:
make
make install clean
As I understand it, the make command takes names of build targets as arguments. So for example install is usually a target that copies some files to standard locations, and clean is a target that removes temporary files.
But what target will make build if no arguments are specified (e.g. the first command in my example)?
By default, it begins by processing the first target that does not begin with a . aka the default goal; to do that, it may have to process other targets - specifically, ones the first target depends on.
The GNU Make Manual covers all this stuff, and is a surprisingly easy and informative read.
To save others a few seconds, and to save them from having to read the manual, here's the short answer. Add this to the top of your make file:
.DEFAULT_GOAL := mytarget
mytarget will now be the target that is run if "make" is executed and no target is specified.
If you have an older version of make (<= 3.80), this won't work. If this is the case, then you can do what anon mentions, simply add this to the top of your make file:
.PHONY: default
default: mytarget ;
References:
https://www.gnu.org/software/make/manual/html_node/How-Make-Works.html
GNU Make also allows you to specify the default make target using a special variable called .DEFAULT_GOAL. You can even unset this variable in the middle of the Makefile, causing the next target in the file to become the default target.
Ref: The Gnu Make manual - Special Variables
bmake's equivalent of GNU Make's .DEFAULT_GOAL is .MAIN:
$ cat Makefile
.MAIN: foo
all:
#echo all
foo:
#echo foo
$ bmake
foo
See the bmake(1) manual page.
I need to track compiler flags used as part of a rule, e.g when supplied as arguments to a function. Does Shake have a way to track such flags as inputs in the same vein as need? As a trivial example, I would like for Shake to rebuild all .o files when the rule changes to pass -O2 to the C compiler instead of -O0.
There are two approaches to tracking things like whether to use optimisation or not.
1) Use Oracles
Oracles very closely match what you are asking for. To track something like
-O0 vs -O2 you'd need an oracle that tracks optimisation level:
newtype OptLevel = OptLevel ()
deriving (Show,Typeable,Eq,Hashable,Binary,NFData)
type instance RuleResult OptLevel = String
rules = do
addOracle $ \(OptLevel _) -> return $
if <whatever you use to decide> then "-O0" else "-O2"
"foo.o" %> \_ -> do
level <- askOracle $ OptLevel ()
cmd "gcc" level ...
Now the optimisation level is a tracked dependency which will update if anything changes. This example is based on the docs for addOracle.
2) Use different output files
For compiler flags another approach is to use different build directories, i.e. so build/opt (and build/opt/obj etc) has binaries and .o files built with -O, build/debug without, build/profile with profiling flags, and build/test with test flags. Some others, like build/doc and build/hsc, have generated files that aren't dependent on compiler flags.
The advantage of this approach is that you can keep all the files cached at once, and refreshing the debug or test build doesn't destroy the opt one.
The disadvantage is that it's just for a set of hardcoded
flags. But it's also not hard to add a new mode, you just need a new
(directory, flags) pair for it.
How can I build one set of source files using two different configurations without having to rebuild everything?
My current setup adds an option --config=rel which will load all options from build_rel.cfg and compile everything to the directory build_rel/.
data Flags = FlagCfg String
deriving (Show, Eq)
flags = [Option ['c'] ["config"]
(ReqArg (\x -> Right $ FlagCfg x) "CFG")
"Specify which configuration to use for the build"]
main :: IO ()
main = shakeArgsWith shakeOptions { shakeChange=ChangeModtimeAndDigest }
flags $
\flags targets -> return $ Just $do
let buildDir = "build" ++
foldr (\a def -> case (a, def) of
(FlagCfg cfg, "") -> '_':cfg
otherwise -> def)
"" flags
-- Settings are read from a config file.
usingConfigFile $ buildDir ++ ".cfg"
...
If I then run
build --config=rel
build --config=dev
I will end up with two builds
build_rel/
build_dev/
However, every time I switch configuration I end up rebuilding everything. I would guess this is because all my oracles have "changed". I would like all oracles to be specific to my two different build directories so that changes will not interfere between builds using different configurations.
I know there is a -m option to specify where the database should be stored but I would rather not have to specify two options that have to sync all the time.
build --config=rel -m build_rel
Is there a way to update the option shakeFiles after the --config option is parsed?
Another idea was to parameterize all my Oracles to include my build configuration but then I noticed that usingConfigFile uses an Oracle and I would have to reimplement that as well. Seems clunky.
Is there some other way to build multiple targets without having to rebuild everything? It seems like such a trivial thing to do but still, I can't figure it out.
There are a few solutions:
Separate databases
If you want the two directories to be entirely unrelated, with nothing shared between them, then changing the database as well makes most sense. There's currently no "good" way to do that (either pass two flags, or pre-parse some of the command line). However, it should be easy enough to add:
shakeArgsOptionsWith
:: ShakeOptions
-> [OptDescr (Either String a)]
-> (ShakeOptions -> [a] -> [String] -> IO (Maybe (ShakeOptions, Rules ())))
-> IO ()
Which would then let you control both settings from a single flag.
Single database
If you want a single database, you could load all the config files, and specify config like release.destination = ... and debug.destination = ..., then rule for */output.txt would lookup the config based on the prefix of the rule, e.g. release/output.txt would look up release.destination. The advantage here is that anything that does not change between debug and release (e.g. documentation) could potentially be shared.
In my Makefiles, I prefer having the output directory defined by a environment variable rather than hard-coded (with some reasonable default value if its unset). For example, a Make rule would look like
$(OUTPUT_DIR)/some_file: deps
#build commands
I have yet to figure out how to achieve a similar goal in Shake. I like using getEnvWithDefault to grab the value of the environment variable or a reasonable default, but no amount of bashing it with binds or lambdas have allowed me to combine it with (*>).
How might it be possible to interpolate an environment variable in a FilePattern for use with (*>)?
The function getEnvWithDefault runs in the Action monad, and the name of the rule has to be supplied in a context where you cannot access the Action monad, so you can't translate this pattern the way you tried. There are a few alternatives:
Option 1: Use lookupEnv before calling shake
To exactly match the behaviour of Make you can write:
main = do
outputDir <- fromMaybe "output" <$> lookupEnv "OUTPUT_DIR"
shakeArgs shakeOptions $ do
(outputDir </> "some_file") *> \out -> do
need deps
-- build commands
Here we use the lookupEnv function (from System.Environment) to grab the environment variable before we start running Shake. We can then define a file that precisely matches the environment variable.
Option 2: Don't force the output in the rule
Alternatively, we can define a rule that builds some_file regardless of what directory it is in, and then use the tracked getEnvWithDefault when we say which file we want to build:
main = shakeArgs shakeOptions $ do
"//some_file" *> \out -> do
need deps
-- build commands
action $ do
out <- getEnvWithDefault "OUTPUT_DIR"
need [out </> "some_file"]
Here the rule pattern can build anything, and the caller picks what the output should be. I prefer this variant, but there is a small risk that if the some_file pattern overlaps in some way you might get name clashes. Introducing a unique name, so all outputs are named something like $OUTPUT_DIR/my_outputs/some_file eliminates that risk, but is usually unnecessary.
I'd like my application to have portable access to the configuration files installed during make install (dist_sysconf_DATA). Is it possible to access $(sysconfdir) via config.h?
It is, but you should not do this according to official voices (as in, I am not gonna search the manual for it now) so as to continue supporting overriding it for specific objects to be built.
make CPPFLAGS="-USYSCONFDIR -DSYSCONFDIR=/blah" thisoneobject.o
Hence, what one is supposed to do:
AM_CPPFLAGS = -DSYSCONFDIR=\"${sysconfdir}\"
If you're using autoheader, adding this to your configure.ac will output a SYSCONFDIR macro in your config.h, and it will be defined with the value $(sysconfdir) or ${prefix}/etc.
if test "x$sysconfdir" = 'x${prefix}/etc'; then
if test "x$prefix" = 'xNONE'; then
sysconfdir=$ac_default_prefix/etc
else
sysconfdir=$prefix/etc
fi
fi
AC_DEFINE_UNQUOTED([SYSCONFDIR], ["$sysconfdir"], [location of system configuration directory])
But I would strongly recommend against doing that, and instead, stick with using the -DSYSCONFDIR flag. It's less code and therefore less prone to something going wrong. Using a condition in configure.ac such I mentioned may not be portable or take into account every case that might be encountered. Using -DSYSCONFDIR is the best option. Sometimes appearance just doesn't matter.
What I believe is most commonly done (and this is what I do)
Add the following in your Makefile.am
AM_CPPFLAGS = -DSYSCONFIR='"$(sysconfdir)"'
And now you can access SYSCONFDIR in source