I am trying to run a biological program called BLASTP which takes in two strings (fasta_GWIDD and fasta_UNIPROT in the code) and compares them. The problem that I am encountering is the use of echo/system in the code. Can anyone suggest what am I missing out??
for(i=0;i<index1;i++)
{
sprintf(fasta_GWIDD,">%s\\n%s\n",fasta_name1[i],fasta_seq1[i]);
setenv("GwiddVar", fasta_GWIDD, 1) ;
sprintf(fasta_UNIPROT,">%s\\n%s\n",fasta_name2[i],fasta_seq2[i]);
setenv("UniprotVar", fasta_UNIPROT, 1) ;
system("blastp -query <(echo -e $GwiddVar) -subject<(echo -e $UniprotVar)");
}
The error is:
sh: -c: line 0: syntax error near unexpected token `('
sh: -c: line 0: `blastp -query <(echo -e $GwiddVar) -subject<(echo -e $UniprotVar)'
It seems that the shell does not understand the
<(echo -e $GwiddVar)
syntax. Mind that the system command may use different shell than the one you are used to (like csh instead of bash, and so on). It's everything in somewhere in your OS config files and profile, but I can't guess what you have out there.
Btw. I think that you should be able to check which shell is being used by the system() command by either of these:
system("echo $SHELL") // should simply write the path to current shell
system("ps -aux") // look at it and find what is the parent of the PS
etc.
Considering that this was correct on some shell:
blastp -query <(echo -e $GwiddVar) -subject<(echo -e $UniprotVar)
The syntax cited above apparently is meant only to pass the variable as intput. I think you are overdoing it. You are using echo -e $GwiddVar to print and capture the data, which you already have in a vairable at hand. Have you tried something as simple as:
blastp -query $GwiddVar -subject $UniprotVar
I don't know which shell you are trying to use, but considering that echo got its data, then it should be exactly the same.
If you are worried about spaces, then various shells usually allow you to use quotation marks:
blastp -query "$GwiddVar" -subject "$UniprotVar"
Of course it depends on the shell. If your program uses a shell that does not like quotation marks, well, you have to adapt it. Not to your shell, but to the shell the system() has used.
Another thing is that using system is quite rough. When you have arguments that are difficult to escape correctly, you should be using other functions like execve that are able to take an array of real raw direct strings and pass them directly as ARGV to the process. Using these, you will not need (and you should not) add any quotes or escape any spaces in the strings to be passed.
sprintf(fasta_GWIDD,">%s\\n%s\n",fasta_name1[i],fasta_seq1[i]);
sprintf(fasta_UNIPROT,">%s\\n%s\n",fasta_name2[i],fasta_seq2[i]);
char** args = .....; // allocate an array of char*[5], malloc, or whatever
args[0] = "blastp";
args[1] = "-query";
args[2] = fasta_GWIDD;
args[3] = "-subject";
args[4] = fasta_UNIPROT;
int errcode = execve(4, args, null);
if( errcode ) ... // check the error (if any) and react
However! Note that the execve comes from the exec family, so it replaces your current process. This is why I write only a sketch and don't show the whole ready-to-run code. You will probably need to fork() before it and then wait for the children in the outer loop.
So, I'd first check the shell and syntax ;)
From man 3 system:
DESCRIPTION
system() executes a command specified in command by calling /bin/sh -c
command, and returns after the command has been completed.
On many systems, /bin/sh is not bash, and even when it is, it is a different configuration of bash (bash typically operates differently if it is invoked as /bin/sh). So, you are passing bash syntax to a shell that is either not bash, or doesn't allow the full set of bash-isms... Also, there's a space missing after -system that might be confusing things as well... And, I'm not entirely sure environment variables are expanded within system() strings...
Related
Is it possible to write a /bin/sh shell script that detects whether or not the underlying shell supports ksh-style arrays? The latter are supported at least in ksh and bash, and in zsh with an option.
Since sh does not allow arrays, and there is no concept of a underlying shell.
Of course it is possible that on your platform, /bin/sh is symlinked to some other shell, or (as it happens on my symstem) is a copy some other shell (i.e. someone has performed a, say, cp /bin/bash /bin/sh).
To test for a symlink, you could do a ls -l /bin/sh and investigate its output.
To test for equality, the best you could do is to loop through the candidate shells you are considering, and test for equality. For instance:
for s in bash ksh
do
if cmp -s /bin/sh /bin/$s
then
echo /bin/sh is in reality a $s
break
fi
done
Of course there is no guaranteed way that it works.
Another possibility would be to run (perhaps using eval) some code fragments which use bash arrays, respectively ksh arrays, and analyze the stderr of it to detect, which ones gives syntax errors.
This question already has answers here:
How can I store a command in a variable in a shell script?
(12 answers)
Closed 4 years ago.
These work as advertised:
grep -ir 'hello world' .
grep -ir hello\ world .
These don't:
argumentString1="-ir 'hello world'"
argumentString2="-ir hello\\ world"
grep $argumentString1 .
grep $argumentString2 .
Despite 'hello world' being enclosed by quotes in the second example, grep interprets 'hello (and hello\) as one argument and world' (and world) as another, which means that, in this case, 'hello will be the search pattern and world' will be the search path.
Again, this only happens when the arguments are expanded from the argumentString variables. grep properly interprets 'hello world' (and hello\ world) as a single argument in the first example.
Can anyone explain why this is? Is there a proper way to expand a string variable that will preserve the syntax of each character such that it is correctly interpreted by shell commands?
Why
When the string is expanded, it is split into words, but it is not re-evaluated to find special characters such as quotes or dollar signs or ... This is the way the shell has 'always' behaved, since the Bourne shell back in 1978 or thereabouts.
Fix
In bash, use an array to hold the arguments:
argumentArray=(-ir 'hello world')
grep "${argumentArray[#]}" .
Or, if brave/foolhardy, use eval:
argumentString="-ir 'hello world'"
eval "grep $argumentString ."
On the other hand, discretion is often the better part of valour, and working with eval is a place where discretion is better than bravery. If you are not completely in control of the string that is eval'd (if there's any user input in the command string that has not been rigorously validated), then you are opening yourself to potentially serious problems.
Note that the sequence of expansions for Bash is described in Shell Expansions in the GNU Bash manual. Note in particular sections 3.5.3 Shell Parameter Expansion, 3.5.7 Word Splitting, and 3.5.9 Quote Removal.
When you put quote characters into variables, they just become plain literals (see http://mywiki.wooledge.org/BashFAQ/050; thanks #tripleee for pointing out this link)
Instead, try using an array to pass your arguments:
argumentString=(-ir 'hello world')
grep "${argumentString[#]}" .
In looking at this and related questions, I'm surprised that no one brought up using an explicit subshell. For bash, and other modern shells, you can execute a command line explicitly. In bash, it requires the -c option.
argumentString="-ir 'hello world'"
bash -c "grep $argumentString ."
Works exactly as original questioner desired. There are two restrictions to this technique:
You can only use single quotes within the command or argument strings.
Only exported environment variables will be available to the command
Also, this technique handles redirection and piping, and other shellisms work as well. You also can use bash internal commands as well as any other command that works at the command line, because you are essentially asking a subshell bash to interpret it directly as a command line. Here's a more complex example, a somewhat gratuitously complex ls -l variant.
cmd="prefix=`pwd` && ls | xargs -n 1 echo \'In $prefix:\'"
bash -c "$cmd"
I have built command processors both this way and with parameter arrays. Generally, this way is much easier to write and debug, and it's trivial to echo the command you are executing. OTOH, param arrays work nicely when you really do have abstract arrays of parameters, as opposed to just wanting a simple command variant.
I would like to use the system() function in C to evaluate an expression within a file cmdfile but I'm not getting the same results as when I do so on the command line directly. The content of cmdfile is the following:
$ cat cmdfile
echo hello
and when I evaluate its content on the command line directly, it works:
$ eval "$(<cmdfile)"
hello
To do the same in C, I'm using system(). This is my code:
$ cat systest.c
#include <stdio.h>
#include <string.h>
int main (int argc, char* argv[])
{
char* cmd = argv[1];
printf("%s\n", cmd);
system(cmd);
return 0;
}
The trouble is that I don't see any output when using the above code:
$ ./systest "eval \"\$(<cmdfile)\""
eval "$(<cmdfile)"
There should be hello printed right after the printf output but it doesn't work. Still, I know that system() is definitely doing something, because if I give it a non-existing filename, dash complains:
$ ./systest "eval \"\$(<cmdfileFF)\""
eval "$(<cmdfileFF)"
sh: 1: cannot open cmdfileFF: No such file
and if I just evaluate echo hello without involving cmdfile, it works too:
$ ./systest "eval \"echo hello\""
eval "echo hello"
hello
I'd like to know what is causing this difference in behaviour. Is there any other way of executing the content of cmdfile in dash? I'm restricted to only using the built-in commands of dash on the command line, so options such as ./systest "eval \"\$(cat cmdfile)\"" are not possible. Further, the expansion of "$(<cmdfile)" should only happen within system(), not before (thus ./systest "eval \"$(<cmdfile)\"" won't work.
I tested this with dash 0.5.10.2-6 and dash 0.5.8-2.1ubuntu2.
Thank you for any insight!
Edit
Thanks to Jonathan Leffler's comment, I now realise that dash doesn't understand the $(<file) syntax. So what would be a dash-compatible equivalent?
Wrap-up
So my confusion was due to the fact that system(...) always uses /bin/sh, but when testing my expressions on the command line, I was accidentally invoking bash instead of dash. Hence the results were different.
$(< …) substitution isn’t POSIX-sh-compatible, but your sh is restricted to about that. A general alternative is to replace < cmdfile with cat cmdfile:
./systest "eval \"\$(cat cmdfile)\""
but I think dot-sourcing is equivalent in this case:
./systest '. ./cmdfile'
The proper fix is to put a shebang line in the script and mark it as executable.
#!/bin/sh
echo "hello"
The shebang needs to be the absolutely first line of the file (its first two bytes should be # and !). The quoting around the argument to echo is not strictly necessary here, but good practice. (See also When to wrap quotes around a shell variable?)
Changing the permissions only needs to be done once, when you have just created the file:
chmod +x ./cmdfile
Now, you can simply use
system("./cmdfile")
This question already has answers here:
How can I store a command in a variable in a shell script?
(12 answers)
Closed 4 years ago.
These work as advertised:
grep -ir 'hello world' .
grep -ir hello\ world .
These don't:
argumentString1="-ir 'hello world'"
argumentString2="-ir hello\\ world"
grep $argumentString1 .
grep $argumentString2 .
Despite 'hello world' being enclosed by quotes in the second example, grep interprets 'hello (and hello\) as one argument and world' (and world) as another, which means that, in this case, 'hello will be the search pattern and world' will be the search path.
Again, this only happens when the arguments are expanded from the argumentString variables. grep properly interprets 'hello world' (and hello\ world) as a single argument in the first example.
Can anyone explain why this is? Is there a proper way to expand a string variable that will preserve the syntax of each character such that it is correctly interpreted by shell commands?
Why
When the string is expanded, it is split into words, but it is not re-evaluated to find special characters such as quotes or dollar signs or ... This is the way the shell has 'always' behaved, since the Bourne shell back in 1978 or thereabouts.
Fix
In bash, use an array to hold the arguments:
argumentArray=(-ir 'hello world')
grep "${argumentArray[#]}" .
Or, if brave/foolhardy, use eval:
argumentString="-ir 'hello world'"
eval "grep $argumentString ."
On the other hand, discretion is often the better part of valour, and working with eval is a place where discretion is better than bravery. If you are not completely in control of the string that is eval'd (if there's any user input in the command string that has not been rigorously validated), then you are opening yourself to potentially serious problems.
Note that the sequence of expansions for Bash is described in Shell Expansions in the GNU Bash manual. Note in particular sections 3.5.3 Shell Parameter Expansion, 3.5.7 Word Splitting, and 3.5.9 Quote Removal.
When you put quote characters into variables, they just become plain literals (see http://mywiki.wooledge.org/BashFAQ/050; thanks #tripleee for pointing out this link)
Instead, try using an array to pass your arguments:
argumentString=(-ir 'hello world')
grep "${argumentString[#]}" .
In looking at this and related questions, I'm surprised that no one brought up using an explicit subshell. For bash, and other modern shells, you can execute a command line explicitly. In bash, it requires the -c option.
argumentString="-ir 'hello world'"
bash -c "grep $argumentString ."
Works exactly as original questioner desired. There are two restrictions to this technique:
You can only use single quotes within the command or argument strings.
Only exported environment variables will be available to the command
Also, this technique handles redirection and piping, and other shellisms work as well. You also can use bash internal commands as well as any other command that works at the command line, because you are essentially asking a subshell bash to interpret it directly as a command line. Here's a more complex example, a somewhat gratuitously complex ls -l variant.
cmd="prefix=`pwd` && ls | xargs -n 1 echo \'In $prefix:\'"
bash -c "$cmd"
I have built command processors both this way and with parameter arrays. Generally, this way is much easier to write and debug, and it's trivial to echo the command you are executing. OTOH, param arrays work nicely when you really do have abstract arrays of parameters, as opposed to just wanting a simple command variant.
What's the difference between "sh -c cmd" and "cmd", when executed from a shell commnad line, and when executed from a c exec() function, respectively? Thanks.
It depends on what 'cmd' represents. In the basic case where it is a simple command name (say ps or ls), then there is no difference at the shell command line, and precious little difference when executed by execvp(). The 'non-p' exec*() functions do have slightly different semantics; they don't use the PATH variable, so the command must exist and be executable in the current directory or it will fail.
However, if cmd is more complex, then it can make a big difference. For example:
$ echo $$
17429
$ sh -c 'echo $$'
76322
$ sh -c "echo $$"
17429
$
The first reports the process ID of the original shell; the second reports the process ID of the shell run as sh; the third is an expensive way of reporting the process ID of the original shell. Note that the single quotes vs double quotes are significant too. Here, the quotes would not be present in the C invocation (the shell removes the quotes from around the arguments), and the value of $$ would be that of the child shell:
char *argv[] = { "sh", "-c", "echo $$", 0 };
execvp(argv[0], argv);
(I said the quotes are not present in the C invocation; they are needed around the string in the C code, but the value passed to sh doesn't contain any quotes — so I meant what I said, though it might not be quite as blindingly obvious as all that.)
From the man page:
-c string If the -c option is present, then commands are read from string. If there are arguments after the string, they are assigned to the positional parameters, starting with $0.
Just cmd will run through bash (or whatever your default) shell is. You need the sh to explicitly call -c argument.
exec shouldn't make a difference here.