What's the difference between "sh -c cmd" and "cmd", when executed from a shell commnad line, and when executed from a c exec() function, respectively? Thanks.
It depends on what 'cmd' represents. In the basic case where it is a simple command name (say ps or ls), then there is no difference at the shell command line, and precious little difference when executed by execvp(). The 'non-p' exec*() functions do have slightly different semantics; they don't use the PATH variable, so the command must exist and be executable in the current directory or it will fail.
However, if cmd is more complex, then it can make a big difference. For example:
$ echo $$
17429
$ sh -c 'echo $$'
76322
$ sh -c "echo $$"
17429
$
The first reports the process ID of the original shell; the second reports the process ID of the shell run as sh; the third is an expensive way of reporting the process ID of the original shell. Note that the single quotes vs double quotes are significant too. Here, the quotes would not be present in the C invocation (the shell removes the quotes from around the arguments), and the value of $$ would be that of the child shell:
char *argv[] = { "sh", "-c", "echo $$", 0 };
execvp(argv[0], argv);
(I said the quotes are not present in the C invocation; they are needed around the string in the C code, but the value passed to sh doesn't contain any quotes — so I meant what I said, though it might not be quite as blindingly obvious as all that.)
From the man page:
-c string If the -c option is present, then commands are read from string. If there are arguments after the string, they are assigned to the positional parameters, starting with $0.
Just cmd will run through bash (or whatever your default) shell is. You need the sh to explicitly call -c argument.
exec shouldn't make a difference here.
Related
I'm studying Apple's implementation of popen() at https://opensource.apple.com/source/Libc/Libc-167/gen.subproj/popen.c.auto.html and noticed that they do execl(_PATH_BSHELL, "sh", "-c", command, NULL) instead of execl(_PATH_BSHELL, command, NULL).
Why would you want to (or should you) exec an executable, e.g. a.out via sh -c instead of just the executable itself?
If you exec sh -c a.out instead of just a.out itself, does the actual a.out process end up being a "grandchild" process and not a child process?
Why would you want to (or should you) exec an executable, e.g. a.out via sh -c instead of just the executable itself?
popen() is designed to run shell commands that include shell syntax like > redirection, | pipes, and && command chaining. It needs to pass the string through sh -c in order to support those constructs. If it didn't those syntactical features would be passed verbatim to the program in question as arguments.
For example, popen("make clean && make") should trigger two make invocations. Without sh -c it would call make once with three arguments, as if one had typed
$ make clean '&&' make
at the terminal.
If you exec sh -c a.out instead of just a.out itself, does the actual a.out process end up being a "grandchild" process and not a child process?
Yes, that is correct. There will be a sh process in between the current process and a.out.
that they do execl(_PATH_BSHELL, "sh", "-c", command, NULL) instead of execl(_PATH_BSHELL, command, NULL)
The latter would NOT have executed command directly, but _PATH_BSHELL (/bin/sh) with its $0 set to command and no arguments, resulting in an shell expecting commands from its stdin.
Also, that syntax relies on NULL being defined to an explicit pointer (e.g. ((void*)0)), and not just 0, which is not guaranteed anywhere. While they can do that in their implementation (because they control all the headers), it's not what you should do in application code.
And no, execl(command, command, (void*)NULL) wouldn't have executed command directly either, unless command is a) a full path and b) in an executable format (binary or a script starting with a she-bang #! -- the latter being a non-standard extension). If command was a simple command name to be looked up in PATH (like pwd or a.out) or an executable script not starting with a she-bang, you should've used execlp instead of execl.
The exec[lv]p[e] functions do some of the things a shell does (like looking through the PATH), but not all of them (like running multiple commands or expanding variables): that's why functions like system(3) or popen(3) pass the command to /bin/sh -c. Notice that with both it's /bin/sh, not the user's login shell or the $SHELL from the environment which is used.
If you exec sh -c a.out instead of just a.out itself, does the actual a.out process end up being a "grandchild" process and not a child process?
Only with some shells like dash. Not with bash, ksh93, mksh, zsh, yash, busybox, etc, which will execute a.out directly instead of forking and waiting for it.
This question already has answers here:
How can I store a command in a variable in a shell script?
(12 answers)
Closed 4 years ago.
These work as advertised:
grep -ir 'hello world' .
grep -ir hello\ world .
These don't:
argumentString1="-ir 'hello world'"
argumentString2="-ir hello\\ world"
grep $argumentString1 .
grep $argumentString2 .
Despite 'hello world' being enclosed by quotes in the second example, grep interprets 'hello (and hello\) as one argument and world' (and world) as another, which means that, in this case, 'hello will be the search pattern and world' will be the search path.
Again, this only happens when the arguments are expanded from the argumentString variables. grep properly interprets 'hello world' (and hello\ world) as a single argument in the first example.
Can anyone explain why this is? Is there a proper way to expand a string variable that will preserve the syntax of each character such that it is correctly interpreted by shell commands?
Why
When the string is expanded, it is split into words, but it is not re-evaluated to find special characters such as quotes or dollar signs or ... This is the way the shell has 'always' behaved, since the Bourne shell back in 1978 or thereabouts.
Fix
In bash, use an array to hold the arguments:
argumentArray=(-ir 'hello world')
grep "${argumentArray[#]}" .
Or, if brave/foolhardy, use eval:
argumentString="-ir 'hello world'"
eval "grep $argumentString ."
On the other hand, discretion is often the better part of valour, and working with eval is a place where discretion is better than bravery. If you are not completely in control of the string that is eval'd (if there's any user input in the command string that has not been rigorously validated), then you are opening yourself to potentially serious problems.
Note that the sequence of expansions for Bash is described in Shell Expansions in the GNU Bash manual. Note in particular sections 3.5.3 Shell Parameter Expansion, 3.5.7 Word Splitting, and 3.5.9 Quote Removal.
When you put quote characters into variables, they just become plain literals (see http://mywiki.wooledge.org/BashFAQ/050; thanks #tripleee for pointing out this link)
Instead, try using an array to pass your arguments:
argumentString=(-ir 'hello world')
grep "${argumentString[#]}" .
In looking at this and related questions, I'm surprised that no one brought up using an explicit subshell. For bash, and other modern shells, you can execute a command line explicitly. In bash, it requires the -c option.
argumentString="-ir 'hello world'"
bash -c "grep $argumentString ."
Works exactly as original questioner desired. There are two restrictions to this technique:
You can only use single quotes within the command or argument strings.
Only exported environment variables will be available to the command
Also, this technique handles redirection and piping, and other shellisms work as well. You also can use bash internal commands as well as any other command that works at the command line, because you are essentially asking a subshell bash to interpret it directly as a command line. Here's a more complex example, a somewhat gratuitously complex ls -l variant.
cmd="prefix=`pwd` && ls | xargs -n 1 echo \'In $prefix:\'"
bash -c "$cmd"
I have built command processors both this way and with parameter arrays. Generally, this way is much easier to write and debug, and it's trivial to echo the command you are executing. OTOH, param arrays work nicely when you really do have abstract arrays of parameters, as opposed to just wanting a simple command variant.
I would like to use the system() function in C to evaluate an expression within a file cmdfile but I'm not getting the same results as when I do so on the command line directly. The content of cmdfile is the following:
$ cat cmdfile
echo hello
and when I evaluate its content on the command line directly, it works:
$ eval "$(<cmdfile)"
hello
To do the same in C, I'm using system(). This is my code:
$ cat systest.c
#include <stdio.h>
#include <string.h>
int main (int argc, char* argv[])
{
char* cmd = argv[1];
printf("%s\n", cmd);
system(cmd);
return 0;
}
The trouble is that I don't see any output when using the above code:
$ ./systest "eval \"\$(<cmdfile)\""
eval "$(<cmdfile)"
There should be hello printed right after the printf output but it doesn't work. Still, I know that system() is definitely doing something, because if I give it a non-existing filename, dash complains:
$ ./systest "eval \"\$(<cmdfileFF)\""
eval "$(<cmdfileFF)"
sh: 1: cannot open cmdfileFF: No such file
and if I just evaluate echo hello without involving cmdfile, it works too:
$ ./systest "eval \"echo hello\""
eval "echo hello"
hello
I'd like to know what is causing this difference in behaviour. Is there any other way of executing the content of cmdfile in dash? I'm restricted to only using the built-in commands of dash on the command line, so options such as ./systest "eval \"\$(cat cmdfile)\"" are not possible. Further, the expansion of "$(<cmdfile)" should only happen within system(), not before (thus ./systest "eval \"$(<cmdfile)\"" won't work.
I tested this with dash 0.5.10.2-6 and dash 0.5.8-2.1ubuntu2.
Thank you for any insight!
Edit
Thanks to Jonathan Leffler's comment, I now realise that dash doesn't understand the $(<file) syntax. So what would be a dash-compatible equivalent?
Wrap-up
So my confusion was due to the fact that system(...) always uses /bin/sh, but when testing my expressions on the command line, I was accidentally invoking bash instead of dash. Hence the results were different.
$(< …) substitution isn’t POSIX-sh-compatible, but your sh is restricted to about that. A general alternative is to replace < cmdfile with cat cmdfile:
./systest "eval \"\$(cat cmdfile)\""
but I think dot-sourcing is equivalent in this case:
./systest '. ./cmdfile'
The proper fix is to put a shebang line in the script and mark it as executable.
#!/bin/sh
echo "hello"
The shebang needs to be the absolutely first line of the file (its first two bytes should be # and !). The quoting around the argument to echo is not strictly necessary here, but good practice. (See also When to wrap quotes around a shell variable?)
Changing the permissions only needs to be done once, when you have just created the file:
chmod +x ./cmdfile
Now, you can simply use
system("./cmdfile")
I am trying to run a biological program called BLASTP which takes in two strings (fasta_GWIDD and fasta_UNIPROT in the code) and compares them. The problem that I am encountering is the use of echo/system in the code. Can anyone suggest what am I missing out??
for(i=0;i<index1;i++)
{
sprintf(fasta_GWIDD,">%s\\n%s\n",fasta_name1[i],fasta_seq1[i]);
setenv("GwiddVar", fasta_GWIDD, 1) ;
sprintf(fasta_UNIPROT,">%s\\n%s\n",fasta_name2[i],fasta_seq2[i]);
setenv("UniprotVar", fasta_UNIPROT, 1) ;
system("blastp -query <(echo -e $GwiddVar) -subject<(echo -e $UniprotVar)");
}
The error is:
sh: -c: line 0: syntax error near unexpected token `('
sh: -c: line 0: `blastp -query <(echo -e $GwiddVar) -subject<(echo -e $UniprotVar)'
It seems that the shell does not understand the
<(echo -e $GwiddVar)
syntax. Mind that the system command may use different shell than the one you are used to (like csh instead of bash, and so on). It's everything in somewhere in your OS config files and profile, but I can't guess what you have out there.
Btw. I think that you should be able to check which shell is being used by the system() command by either of these:
system("echo $SHELL") // should simply write the path to current shell
system("ps -aux") // look at it and find what is the parent of the PS
etc.
Considering that this was correct on some shell:
blastp -query <(echo -e $GwiddVar) -subject<(echo -e $UniprotVar)
The syntax cited above apparently is meant only to pass the variable as intput. I think you are overdoing it. You are using echo -e $GwiddVar to print and capture the data, which you already have in a vairable at hand. Have you tried something as simple as:
blastp -query $GwiddVar -subject $UniprotVar
I don't know which shell you are trying to use, but considering that echo got its data, then it should be exactly the same.
If you are worried about spaces, then various shells usually allow you to use quotation marks:
blastp -query "$GwiddVar" -subject "$UniprotVar"
Of course it depends on the shell. If your program uses a shell that does not like quotation marks, well, you have to adapt it. Not to your shell, but to the shell the system() has used.
Another thing is that using system is quite rough. When you have arguments that are difficult to escape correctly, you should be using other functions like execve that are able to take an array of real raw direct strings and pass them directly as ARGV to the process. Using these, you will not need (and you should not) add any quotes or escape any spaces in the strings to be passed.
sprintf(fasta_GWIDD,">%s\\n%s\n",fasta_name1[i],fasta_seq1[i]);
sprintf(fasta_UNIPROT,">%s\\n%s\n",fasta_name2[i],fasta_seq2[i]);
char** args = .....; // allocate an array of char*[5], malloc, or whatever
args[0] = "blastp";
args[1] = "-query";
args[2] = fasta_GWIDD;
args[3] = "-subject";
args[4] = fasta_UNIPROT;
int errcode = execve(4, args, null);
if( errcode ) ... // check the error (if any) and react
However! Note that the execve comes from the exec family, so it replaces your current process. This is why I write only a sketch and don't show the whole ready-to-run code. You will probably need to fork() before it and then wait for the children in the outer loop.
So, I'd first check the shell and syntax ;)
From man 3 system:
DESCRIPTION
system() executes a command specified in command by calling /bin/sh -c
command, and returns after the command has been completed.
On many systems, /bin/sh is not bash, and even when it is, it is a different configuration of bash (bash typically operates differently if it is invoked as /bin/sh). So, you are passing bash syntax to a shell that is either not bash, or doesn't allow the full set of bash-isms... Also, there's a space missing after -system that might be confusing things as well... And, I'm not entirely sure environment variables are expanded within system() strings...
I have a command that execute well in the normal terminal on Linux:
xterm -e bash -c "some commands"
I want to execute the above command using c program execXX system calls. I try to use the following codes but it gives me a normal xterm window.
execl("/usr/bin/xterm", "/usr/bin/xterm -e bash -c \"some commands\"", NULL);
Is there any way I can execute the above command using execXX system calls? Thank you!
You need to call it like:
execl("/usr/bin/xterm", "/usr/bin/xterm", "-e", "bash", "-c", "some commands", (void*)NULL);
The convention is to let the first argument be the same as the path to the program. If you have spaces in the arguments, it will be the same effect as calling xterm 'something with spaces' instead of xterm something with spaces.
A possible tangent: is there any reason why you need these to run specifically within xterm? If you just want to run some shell commands, then running them within /bin/sh or /bin/bash would be more natural, and probably more reliable.