CMake: how to avoid escaping spaces in command line? [duplicate] - c

I'm trying to create a custom command that runs with some environment variables, such as LDFLAGS, whose value needs to be quoted if it contains spaces:
LDFLAGS="-Lmydir -Lmyotherdir"
I cannot find a way to include this argument in a CMake custom command, due to CMake's escaping rules. Here's what I've tried so far:
COMMAND LDFLAGS="-Ldir -Ldir2" echo blah VERBATIM)
yields "LDFLAGS=\"-Ldir -Ldir2\"" echo blah
COMMAND LDFLAGS=\"-Ldir -Ldir2\" echo blah VERBATIM)
yields LDFLAGS=\"-Ldir -Ldir2\" echo blah
It seems I either get the whole string quoted, or the escaped quotes don't resolve when used as part of the command.
I would appreciate either a way to include the literal double-quote or as an alternative a better way to set environment variables for a command. Please note that I'm still on CMake 2.8, so I don't have the new "env" command available in 3.2.
Note that this is not a duplicate of When to quote variables? as none of those quoting methods work for this particular case.

The obvious choice - often recommended when hitting the boundaries of COMMAND especially with older versions of CMake - is to use an external script.
I just wanted to add some simple COMMAND only variations that do work and won't need a shell, but are - I have to admit - still partly platform dependent.
One example would be to put only the quoted part into a variable:
set(vars_as_string "-Ldir -Ldir2")
add_custom_target(
QuotedEnvVar
COMMAND env LD_FLAGS=${vars_as_string} | grep LD_FLAGS
)
Which actually does escape the space and not the quotes.
Another example would be to add it with escaped quotes as a "launcher" rule:
add_custom_target(
LauncherEnvVar
COMMAND env | grep LD_FLAGS
)
set_target_properties(
LauncherEnvVar
PROPERTIES RULE_LAUNCH_CUSTOM "env LD_FLAGS=\"-Ldir -Ldir2\""
)
Edit: Added examples for multiple quoted arguments without the need of escaping quotes
Another example would be to "hide some of the complexity" in a function and - if you want to add this to all your custom command calls - use the global/directory RULE_LAUNCH_CUSTOM property:
function(set_env)
get_property(_env GLOBAL PROPERTY RULE_LAUNCH_CUSTOM)
if (NOT _env)
set_property(GLOBAL PROPERTY RULE_LAUNCH_CUSTOM "env")
endif()
foreach(_arg IN LISTS ARGN)
set_property(GLOBAL APPEND_STRING PROPERTY RULE_LAUNCH_CUSTOM " ${_arg}")
endforeach()
endfunction(set_env)
set_env(LDFLAGS="-Ldir1 -Ldir2" CFLAGS="-Idira -Idirb")
add_custom_target(
MultipleEnvVar
COMMAND env | grep -E 'LDFLAGS|CFLAGS'
)
Alternative (for CMake >= 3.0)
I think what we actually are looking for here (besides the cmake -E env ...) is named Bracket Argument and does allow any character without the need of adding backslashes:
set_property(
GLOBAL PROPERTY
RULE_LAUNCH_CUSTOM [=[env LDFLAGS="-Ldir1 -Ldir2" CFLAGS="-Idira -Idirb"]=]
)
add_custom_target(
MultipleEnvVarNew
COMMAND env | grep -E 'LDFLAGS|CFLAGS'
)
References
0005145: Set environment variables for ADD_CUSTOM_COMMAND/ADD_CUSTOM_TARGET
How to modify environment variables passed to custom CMake target?
[CMake] How to set environment variable for custom command
cmake: when to quote variables?

You need three backslashes. I needed this recently to get a preprocessor define from PkgConfig and apply it to my C++ flags:
pkg_get_variable(SHADERDIR movit shaderdir)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -DSHADERDIR=\\\"${SHADERDIR}\\\"")

Florian's answer is wrong on several counts:
Putting the quoted part in a variable makes no difference.
You should definitely use VERBATIM. It fixes platform-specific quoting bugs.
You definitely shouldn't use RULE_LAUNCH_CUSTOM for this. It isn't intended for this and only works with some generators.
You shouldn't use env as the command. It isn't available on Windows.
It turns out the real reason OPs code doesn't work is that CMake always fully quotes the first word after COMMAND because it's supposed to be the name of an executable. You simply shouldn't put environment variables first.
For example:
add_custom_command(
OUTPUT q1.txt
COMMAND ENV_VAR="a b" echo "hello" > q1.txt
VERBATIM
)
add_custom_target(q1 ALL DEPENDS q1.txt)
$ VERBOSE=1 make
...
"ENV_VAR=\"a b\"" echo hello > q1.txt
/bin/sh: ENV_VAR="a b": command not found
So how do you pass an environment variable with spaces? Simple.
add_custom_command(
OUTPUT q1.txt
COMMAND ${CMAKE_COMMAND} -E env ENV_VAR="a b" echo "hello" > q1.txt
VERBATIM
)

Ok, I removed my original answer as the one proposed by #Florian is better. There is one additional tweak needed for multiple quoted args. Consider a list of environment variables as such:
set(my_env_vars LDFLAGS="-Ldir1 -Ldir2" CFLAGS="-Idira -Idirb")
In order to produce the desired expansion, convert to string and then replace ; with a space.
set(my_env_string "${my_env_vars}") #produces LDFLAGS="...";CFLAGS="..."
string(REPLACE ";" " " my_env_string "${my_env_string}")
Then you can proceed with #Florian's brilliant answer and add the custom launch rule. If you need semicolons in your string then you'll need to convert them to something else first.
Note that in this case I didn't need to launch with env:
set_target_properties(mytarget PROPERTIES RULE_LAUNCH_CUSTOM "${my_env_string}")
This of course depends on your shell.
On second thought, my original answer is below as I also have a case where I don't have access to the target name.
set(my_env LDFLAGS=\"-Ldir -Ldir2" CFLAGS=\"-Idira -Idirb\")
add_custom_command(COMMAND sh -c "${my_env} grep LDFLAGS" VERBATIM)
This technique still requires that the semicolons from the list->string conversion be replaced.

Some folks suggest to use ${CMAKE_COMMAND} and pass your executable as an argument, e.g:
COMMAND ${CMAKE_COMMAND} -E env "$(WindowsSdkDir)/bin/x64/makecert.exe" ...
That worked for me.

Related

Bash running explicit path works, but variable path doesn't [duplicate]

This question already has answers here:
How can I store a command in a variable in a shell script?
(12 answers)
Closed 4 years ago.
These work as advertised:
grep -ir 'hello world' .
grep -ir hello\ world .
These don't:
argumentString1="-ir 'hello world'"
argumentString2="-ir hello\\ world"
grep $argumentString1 .
grep $argumentString2 .
Despite 'hello world' being enclosed by quotes in the second example, grep interprets 'hello (and hello\) as one argument and world' (and world) as another, which means that, in this case, 'hello will be the search pattern and world' will be the search path.
Again, this only happens when the arguments are expanded from the argumentString variables. grep properly interprets 'hello world' (and hello\ world) as a single argument in the first example.
Can anyone explain why this is? Is there a proper way to expand a string variable that will preserve the syntax of each character such that it is correctly interpreted by shell commands?
Why
When the string is expanded, it is split into words, but it is not re-evaluated to find special characters such as quotes or dollar signs or ... This is the way the shell has 'always' behaved, since the Bourne shell back in 1978 or thereabouts.
Fix
In bash, use an array to hold the arguments:
argumentArray=(-ir 'hello world')
grep "${argumentArray[#]}" .
Or, if brave/foolhardy, use eval:
argumentString="-ir 'hello world'"
eval "grep $argumentString ."
On the other hand, discretion is often the better part of valour, and working with eval is a place where discretion is better than bravery. If you are not completely in control of the string that is eval'd (if there's any user input in the command string that has not been rigorously validated), then you are opening yourself to potentially serious problems.
Note that the sequence of expansions for Bash is described in Shell Expansions in the GNU Bash manual. Note in particular sections 3.5.3 Shell Parameter Expansion, 3.5.7 Word Splitting, and 3.5.9 Quote Removal.
When you put quote characters into variables, they just become plain literals (see http://mywiki.wooledge.org/BashFAQ/050; thanks #tripleee for pointing out this link)
Instead, try using an array to pass your arguments:
argumentString=(-ir 'hello world')
grep "${argumentString[#]}" .
In looking at this and related questions, I'm surprised that no one brought up using an explicit subshell. For bash, and other modern shells, you can execute a command line explicitly. In bash, it requires the -c option.
argumentString="-ir 'hello world'"
bash -c "grep $argumentString ."
Works exactly as original questioner desired. There are two restrictions to this technique:
You can only use single quotes within the command or argument strings.
Only exported environment variables will be available to the command
Also, this technique handles redirection and piping, and other shellisms work as well. You also can use bash internal commands as well as any other command that works at the command line, because you are essentially asking a subshell bash to interpret it directly as a command line. Here's a more complex example, a somewhat gratuitously complex ls -l variant.
cmd="prefix=`pwd` && ls | xargs -n 1 echo \'In $prefix:\'"
bash -c "$cmd"
I have built command processors both this way and with parameter arrays. Generally, this way is much easier to write and debug, and it's trivial to echo the command you are executing. OTOH, param arrays work nicely when you really do have abstract arrays of parameters, as opposed to just wanting a simple command variant.

How to "-#-expand ("${array[#]}") possibly empty array in Bash when nounset is set?

I have bash script with set -o nounset option (and I want that!).
Now, I want construct a command invocation, but I don't know number of arguments beforehand, so I want to use an array for that (example below). However, when ARRAY is an empty array, "${ARRAY[#]}" fails.
Question: how to #-expand array ("${ARRAY[#]}") so that the expansion does not fail when set -o nounset is on?
Example:
# Clone git repo. Use --reference if ${reference_local_repo} exist.
reference_local_repo=.....
test -d "${reference_local_repo}" \
&& reference=("--reference" "${reference_local_repo}") \
|| reference=()
git clone "${reference[#]}" http://address/of/the/repo
Of course, I could use the following instead:
# bad example
reference=''
test -d "${reference_local_repo}" && reference="--reference ${reference_local_repo}"
... but that wouldn't work if the path to local repo contained a whitespace.
As a workaround, instead of reference=() i use reference=("-c" "dummy.dummy=dummy"). That way I avoid empty array, and Bash does not complain. Alternatively, i can (rename the array variable and) have "clone" as the first array element. So I got this working, but I'd like to learn The Proper Way.
For the record, I'm using GNU bash, version 4.3.42(1)-release (x86_64-pc-linux-gnu).
To answer your specific question: The very old and simple way to deal with this is:
${reference[#]+"${reference[#]}"}
If reference is unset, nothing is expanded.
If it is set, all its components are expanded.
Read the historical roots for this use:
Once upon 20 or so years ago, some broken minor variants of the Bourne Shell substituted an empty string "" for "$#" if there were no arguments,
Of course, in this specific case:
test -d "${reference_local_repo}" && abool="" || unset abool
git clone ${abool+--reference "$reference_local_repo"} http://address/of/the/repo
When abool is set to NUL ("") (or some other value if you so choose to use), it is set, and in the next line it expands to what is after the plus (yes, as exactly two parameters).
When abool is unset, it completely disappears in the next line expansion.
Maybe this is more verbose:
unset abool
if test -d "${reference_local_repo}"; then abool="ValidDir"; fi
git clone ${abool+--reference "$reference_local_repo"} http://address/of/the/repo
I don't understand why you're using an array here. You could just:
test -d "${reference_local_repo}" \
&& reference="${reference_local_repo}" \
|| reference=""
git clone ${reference:+--reference "$reference"} http://address/of/the/repo
Now there are no undefined variables, and no mucking about with arrays for what is actually a single value.
You may use an auxiliar variable (or just redefine the same variable) to check if an array has anything:
foo=${your_array[#]:-}
and then:
git clone ... "${foo}" ...
This is compatible with the nounset flag. The :- expansion at the end of the variable (${your_array[#]:-}) will yield an empty string if $your_array is undefined.

Bash extended globbing brackets break array initalization

I used the statement echo *([!min]).css to get all filenames in the current directory with the .css extension, except for the ones with .min.css extension. That worked on the bash.
However, when I use this to initialize an array in a bash script like that
files=(*([!min]).css)
it doesn't work anymore. Bash says there is an unexpected opening bracket somewhere. My editor's syntax highlighting also looks like the brackets of the glob inside the array initialization are not correct, however I wasn't able to get it right.
Any advice? Thanks.
EDIT: I use GNU Bash 4.3.033 on ArchLinux.
To use extended globs, you must enable the extglob shell option. Put it at the start of your script, just below the shebang:
#!/usr/bin/env bash
shopt -s extglob
#...
files=( !(*.min).css )
#...
Note that shell options are not inherited, so even though you may have extglob enabled in the interactive bash you run the script from, you still have to explicitly enable it in the script.

How do I put basename into a variable?

#!/bin/bash
file=debian.deb
test=basename $file .deb
DP="blah/blah/$test/$test.php"
read -p "[$DP]: " DPREPLY
DPREPLY=${DPREPLY:-$DP}
echo "Blah is set to $DPREPLY"
echo $DPREPLY>>testfile
So what I'm trying to do is set the variable test from the variable file and use it in the file testfile.
Use the command substitution $(...) mechanism:
test=$(basename "$file" .deb)
You can also use backquotes, but these are not recommended in modern scripts (mainly because they don't nest as well as the $(...) notation).
test=`basename "$file" .deb`
You need to know about backquotes in order to interpret other people's scripts; you shouldn't be using them in your own.
Note the use of quotes around "$file"; this ensures that spaces in filenames are handled correctly.

How to include pipe character in an argument to a batch file from a bash script?

I have a shell script that I want to execute this line:
qtvars.bat vsstart "qt.sln" /BUILD "Debug|Win32"
This works fine (though I had to modify qtvars.bat, but that's beside the point). The problem is that I want the command to execute to be in a variable:
EDIT: This doesn't work either, if I type it into bash. Previously I was typing it into cmd.exe, which hardly made for a fair comparison.
command="qtvars.bat"
args="vsstart"
$command $args "qt.sln" /BUILD "Debug|Win32"
Now it chokes on the pipe! I get this message:
'Win32' is not recognized as an internal or external command,
operable program or batch file.
I've tried a bunch of forms of escaping the quotes and/or pipe, all to no avail. Interestingly, it works when it's an executable rather than a batch file, e.g.:
command="devenv.exe"
args=""
$command $args "qt.sln" /BUILD "Debug|Win32"
Thanks for any ideas.
I know you "escape" the pipe character in a batch file with the ^ character, so...
echo ^| Some text here ^|
Would display...
| Some text here |
I don't know whether that would help you in this instance? Maybe try prepending each pipe character with a ^ and see what happens? :-)
This is a classic case of double-escaping, where both bash and CMD.EXE need to be instructed to ignore the special | (pipe) character.
Try the following:
$command $args "qt.sln" /BUILD '"Debug|Win32"'
This will be the equivalent of you typing, at a CMD.EXE prompt:
qtvars.bat vsstart qt.sln /BUILD "Debug|Win32"
Using the above, you are essentially forcing the passing of the double-quotes on to CMD.EXE (instead of bash eating them away.) The outermost single quotes instruct bash not to interpret or touch in any way what's inside them; the inner double-quotes instruct CMD.EXE to ignore any special characters (the pipe in this case) within.
Alternatively, you can also try:
$command $args "qt.sln" /BUILD 'Debug\|Win32'
This should be the equivalent of you typing, at a CMD.EXE prompt:
qtvars.bat vsstart qt.sln /BUILD Debug\|Win32
Note the use of single quotes (!), which ensure that bash will not interpret the \ (and, instead, will pass it as-is to CMD.EXE.)
Here's another solution (workaround?) I've found:
first, ensure an environment variable defines the pipe character, for example:
set PIPE="|"
later, run the command specifying the above defined environment variable name:
"c:\(...)\devenv.com" foo.sln /build Debug%PIPE%Win32
That does the job even if there are multiple wrappers between the caller and the callee. I'm now using it with a very long chain of wrappers:
Python/Linux -> VirtualBox guest's executeProcess -> Cmd/Windows -> devenv.com
(cross posted to: How to pass a quoted pipe character to cmd.exe?)
Escaping a piping character in the Windows scripting language is done with a caret (^). I just had to do this the other day. I know this is old, but I thought I would post what I found in case others ran across this, like I did.
I'd consider going the easy route, and passing a placeholder-token instead - "$P", and then replace it within the CMD/Batch file; e.g. using the 'UnxUtils' SEd command to do the replacement:
For /F "usebackq delims=" %%r in (`Echo %Cmd% ^| sed -e "s/$P/|/g"`) do #Set Cmd2=%%r
REM Now run the command, with the proper pipe symbol in place
%Cmd2%
So having passed the command arg/CMD script args - "git status $P wc -l".
Interesting! What does escaping the | do?
Do these work?
echo "Debug|Win32"
echo "qt.sln" /BUILD 'Debug|Win32'

Resources