I use fork()/exec()/wait() rather than system() when the command has user input as some of its arguments so the user can't put something like...
&rm -rf /home/* && echo HAHA
... as an argument.
I'm assuming popen is as dangerous as system() because it takes a single string and not a list of strings like the exec family of functions do.
I can only get the return value from the exec functions though. Is there a "safe" version of popen that I can run with user input and process stdout / stderr back in the parent process?
The safe way is to set up the necessary pipes yourself, using straight pipe() calls directly.
That's what popen() does under the hood, but it also invokes the shell in order to run the child process. Skipping that step should make it safer.
Related
How do I redirect stderr (or stdout+stderr) to a file if I don't know which shell (bash, csh, dash) is interpreting my command?
My C code running on Linux/FreeBSD/OSX needs to call an external program via the system() function, which will use /bin/sh to interpret the supplied command line. I would like to capture the messages printed by that external program to stderr and save them to a file. The problem is that on different systems /bin/sh points to different shells that have different syntax for redirecting the stderr stream to a file.
The closest thing I found is that bash actually understands the csh-style syntax for redirecting stderr+stdout to a file:
some_program >& output.txt
but dash, which is the default shell on Ubuntu (i.e. very common), does not understand this syntax.
Is there a syntax for stderr redirection that would be correctly interpreted by all common shells? Alternatively, is there a way to tell system() (or some other similar C function?) to use /usr/bin/env bash instead of /bin/sh to interpret the supplied command line?
You have a mistaken assumption, that /bin/sh can be an "alternate" shell like csh that's incompatible with the standard shell syntax. If you had a system setup like that, it would be unusably broken; no shell scripts would work. Pretty much all modern systems attempt to conform, at least superficially, to the POSIX standard, where the sh command processes the Shell Command Language specified in POSIX, which is roughly equivalent to the historical Bourne shell and which bash, dash, ash, etc. (shells which are commonly installed as /bin/sh) are all 99.9% compatible with.
You can completely ignore csh and similar. They're never installed as sh, and only folks who actually want to use them, or who get stuck using them as their interactive shell because some evil sysadmin setup the login shell defaults that way, ever have to care about them.
On any POSIX-like system, you can use
system("some_program > output.txt 2>&1");
This is because POSIX system is equivalent to calling sh, and POSIX sh supports this kind of redirection. This works independently of whether or not a user opening a terminal on the system will see a Csh prompt.
How do I redirect stderr (or stdout+stderr) to a file if I don't know which shell (bash, csh, dash) is interpreting my command?
You don't. Bourne-family shells and csh-family shells have different, incompatible syntax for redirecting stderr. In fact, csh and tcsh do not have a syntax to redirect only stderr at all -- they can redirect it only together with stdout.
If you really could be in any shell at all, then you're pretty much hosed with respect to doing much of anything. One could imagine an obscure, esoteric shell with completely incompatible syntax. For that matter, even an unusual configuration of a standard shell could trip you up -- for example if the IFS variable is set to an unusual value in a Bourne-family shell, then you'll have trouble executing any commands that don't take that into account.
If you can count on executing at least simple commands, then you could execute a known shell within the unknown one to process your command, but that oughtn't to be necessary for the case that seems to interest you.
Alternatively, is there a way to tell system() (or some other similar
C function?) to use /usr/bin/env bash instead of /bin/sh to interpret
the supplied command line?
Not on a POSIX-conforming system. POSIX specifies explicitly that the system() function executes the command by use of /bin/sh -c [the_command]. But this shouldn't be something to worry about, as /bin/sh should be a conforming POSIX shell, or at least pretty close to one. Definitely it should be a Bourne-family shell, which both bash and dash are, but tcsh most definitely is not.
The way to redirect the standard error stream in a POSIX shell is to use the 2> redirection operator (which is a special case of a more general redirection feature applicable to any file descriptor). Whatever shell /bin/sh actually is should recognize that syntax, and in particular bash and dash both do:
some_program 2> output.txt
I think, there is another possibility worth mentioning: You could open the file you want to redirect on stderr in your c-code prior to calling system(). You can dup() the original stderr first, and then restore it again.
fflush(stderr); // Flush pending output
int saved_stderr = dup(fileno(stderr));
int fd = open("output.txt", O_RDWR|O_CREAT|O_TRUNC, 0600);
dup2(fd, fileno(stderr));
close(fd);
system("some_program");
dup2(saved_stderr, fileno(stderr));
close(saved_stderr);
This should perform the output redirection as you need it.
If you don't know the shell.... of course you don't know how to redirect from it, despite of the fact that you can see what value the $SHELL has, and act in consequence:
char *shell = getenv("SHELL");
if (*shell) { /* no SHELL variable defined */
/* ... */
} else if (!strcmp(shell, "/bin/sh")) { /* bourne shell */
/* ... */
} /* ... more shells */
Despite of what you say in your question, it is quite unusual to rename /bin/sh to use another shell, as shell scripts use syntax that depends on that. The only case I know is with bash(1), and I have seen this only in Linux (and remarkably, last versions of solaris), but the syntax of bash(1) is a superset of the syntax of sh(1), making it possible to run shell scripts made for sh(1) with it. Renaming /bin/sh to perl for example, would make your system probably completely unusable, as many system tools depend of /bin/sh to be a bourne compatible shell.
By the way, the system(3) library function always calls sh(1) as the command interpreter, so there should be no problem to use it, but there's no solution to capture the output and process it by the parent process (indeed, the parent process is the sh(1) that system(3) fork(2)s)
Another thing you can do is to popen(3) a process. This call gives you a FILE pointer to a pipe of a process. You popen its input in case you popen(3) it for writing, and you popen its output if you want or read its output. Look at the manual for details, as I don't know now if it redirects only its standard output or it also redirects the standard error (I think only redirects standard output, for reasons discussed below, and only if you popen(3) it with a "r" flag).
FILE *f_in = popen("ps aux", "r");
/* read standard output of 'ps aux' command. */
pclose(f_in); /* closes the descriptor and waits for the child to finish */
Another thing you can do is to redirect yourself after fork(2)ing the child, and before the exec(2) call (this way you can decide if you want only stdout or if you want also stderr redirected back to you):
int fd[2];
int res = pipe(fd);
if (res < 0) {
perror("pipe");
exit(EXIT_FAILURE);
}
if ((res = fork()) < 0) {
perror("fork");
exit(EXIT_FAILURE);
} else if (res == 0) { /* child process */
dup2(fd[1], 1); /* redirect pipe to stdout */
dup2(fd[1], 2); /* redirect pipe also to stderr */
close(fd[1]); close(fd[0]); /* we don't need these */
execvp(program, argv);
perror("execvp");
exit(EXIT_FAILURE);
} else { /* parent process */
close(fd[1]); /* we are not going to write in the pipe */
FILE *f_in = fdopen(fd[0]);
/* read standard output and standard error from program from f_in FILE descriptor */
fclose(f_in);
wait(NULL); /* wait for child to finish */
}
You can see a complete example of this (not reading standard error, but it is easy to add --- you have only to add the second dup2() call from above) here. The program executes repeatedly a command you pass to it on the command line. It needs to get access to the output of the subprocess to count the lines, as between invocations, the program goes up as many lines as the program output, to make the next invocation to overlap the output of the last invocation. You can try it and play, making modifications as you like.
NOTE
In your sample redirection, when you use >&, you need to add a number after the ampersand, to indicate which descriptor you are dup()ing. As the number before the > is optional, the one after the & is mandatory. So, if you have not used it, prepare to receive an error (which probably you don't see if you are redirecting stderr) The idea of having two separate output descriptors is to allow you to redirect stdout and at the same time, conserve a channel where to put error messages.
I'm trying to write a shell that will eventually take advantage of concurrency. Right now I've got a working shell parser but I'm having trouble figuring out how to execute commands. I've looked a bit at exec (execvp etc.) and it looks promising but I have some questions.
Can exec handle file input/output redirection? Can I set up pipes using exec?
I'm also wondering about subshells. What should subshells return; the exit status of its last statement? Can subshells be a part of a pipe?
These might seem like really dumb questions but please bear with my inexperience.
Can exec handle file input/output redirection?
No, you do that with open() and dup() or dup2() (and close()).
Can I set up pipes using exec?
No, you do that with pipe(), dup() or dup2() and lots of close() calls.
I'm also wondering about subshells. What should subshells return, the exit status of its last statement?
That's the normal convention, yes.
Can subshells be a part of a pipe?
Yes. In a normal shell, you can write something like:
(cd /some/where; find . -name '*.png') | sed 's/xyz/prq/' > mapped.namelist
If you want to get scared, you could investigate posix_spawn() and its support functions. Search for 'spawn' at the POSIX 2008 site, and be prepared to be scared. I think it is actually easier to do the mapping work ad hoc than to codify it using posix_spawn() and its supporters.
The standard technique for a shell is to use fork-exec. In this model, to execute an application the shell uses fork to create a new process that is a copy of itself, and then uses one of the exec variants to replace its own code, data, etc. with the information specified by an executable file on disk.
The nice thing about this model is that the shell can do a little extra work with file descriptors (which are not thrown away by exec) before it changes out its address space. So to implement redirection, it changes out the file descriptors 0, 1, and 2 (stdin, stdout, and stderr, respectively) to point to another open file, instead of console I/O. You use dup2 to change out the meaning of one of these file descriptors, and then exec to start the new process.
If I ran the ssh command and logged into a server, would there be any reason to have code call system() since I can run it myself?
Edit: The code I have would be written in C
system call will execute the program (with parameters) that you want. A system() call invokes a shell. So from inside of a C program, if you want to remove a file, you can invoke system with "rm filename" as argument (this is just a use case - definitely not how you'd like to delete a file from a C program)
You should use system() only when you know what you are doing. If a user input is any part of the argument to the system call, you should make sure you are sanitizing your input lest you are opening yourself to command injections.
An example of a command injections with system call is here
Alternatives to system are popen and obviously fork+exec.
I have C program ( program.c ) that calls a shell script ( command.sh ).
command.sh returns an output (a password), i need to get this output in my program.c.
using system(command); , i can't get hold of the output.
is there any other way in C to solve my problem?
Not in pure C. You want POSIX. Specifically popen function.
The specification has a nice example, that you can just copy 1:1
http://pubs.opengroup.org/onlinepubs/009604499/functions/popen.html
Sounds like you're afraid to use libraries. Please try and use libraries, they're just as much part of Unix as shell tools.
In pure C (well ... ignoring the contents of the system() call) you can redirect the shell script output to a file, then read that file.
system("whatever > file");
handle = fopen("file", "r");
/* if ok use handle */
fclose(handle);
You can use popen() as suggested above, but note that in this case you have no control over the process you created (e.g. you will not be able to kill it), you also will not know its exit status.
I suggest using classical pipe/fork/exec combination. Then you will know the pid of your child process, so you will be able so send signals, also with pipe() you are able to redirect process standard output, so you can easily read it in your parent process. As example you can see my accepted answer to popen() alternative.
You should open a pipe using popen but this sounds tedious. A C program calling a shell script for a password.
I have some problem to figure out how I can maintain the pipe and redirect functionality of a shell once I find out that there are missing command line arguments.
If I for example use a scanf call, that will work with a re-direct or a pipe from a shell, but in absence of this I get a prompt, which I don't want.
I would like to accept command line arguments through argv[], a pipe or re-direct but I can't figure out how to do it with out getting the prompt. If I for example try something like this:
if(argc < 2)
exit(0);
Then the program will terminate if I try this:
echo arg | myProgram
Or this:
myProgram < fileWithArgument
I have tried to look this up but I always get some bash scripting reference.
The common way to handle situations like this is to check if the standard input stream is connected to a terminal or not, using isatty or similar functions depending on your OS. If it is, you take parameters from the command line, if not (it's been redirected), you read standard input.
Short version: You can't do it.
Pipeline and redirect specifiers are not arguments to your program, rather they are commands to the invoking shell and are processed before the running instance of your program even exists. The shell does no pass them to the program in argv or any other variable, and you can not discover them in any reliable way.
Neil has given you the way to determine if you are connected to a terminal.
In your examples you are using pipe redirection, both echo arg | myProgram and myProgram < filesWithArguments are sending output to the STDIN of your program.
If you want to read these values, use scanf or fread on the STDIN file descriptor.
If you are trying to get the file content as an argument list for your executable, you need to use it like this:
# This will pass `lala` as a variable
myProgram `echo lala`