I have a project the requires the use of the exec family. My project consist of making an interactive shell. The shell will implement a few basic commands like cd, ls, echo, etc. I have been researching the use of exec, but have not found a useful site. Any suggested links would help.
int ret;
ret = execl ("/bin/ls", "ls", "-1", (char *)0);
How would i get the output of this operation to show on the screen?
doing
int fd = 1;
dup(fd);
close(fd);
gets the output to the screen.
The code you wrote works for me in a simple test program that does nothing else. Remember, when you call execl, the process retains all of the old file handles. So whatever stdout was when you call execl, it will be the same when the new binary is loaded. If you just want the output to go to the terminal, just make sure stdout goes to the terminal.
If you want to do I/O with another program, popen is good for this (as mgb mentioned). It will fork a new process, set up plumbing for you, call some variant of exec, and return a file handle you can use for communication.
Related
How do I redirect stderr (or stdout+stderr) to a file if I don't know which shell (bash, csh, dash) is interpreting my command?
My C code running on Linux/FreeBSD/OSX needs to call an external program via the system() function, which will use /bin/sh to interpret the supplied command line. I would like to capture the messages printed by that external program to stderr and save them to a file. The problem is that on different systems /bin/sh points to different shells that have different syntax for redirecting the stderr stream to a file.
The closest thing I found is that bash actually understands the csh-style syntax for redirecting stderr+stdout to a file:
some_program >& output.txt
but dash, which is the default shell on Ubuntu (i.e. very common), does not understand this syntax.
Is there a syntax for stderr redirection that would be correctly interpreted by all common shells? Alternatively, is there a way to tell system() (or some other similar C function?) to use /usr/bin/env bash instead of /bin/sh to interpret the supplied command line?
You have a mistaken assumption, that /bin/sh can be an "alternate" shell like csh that's incompatible with the standard shell syntax. If you had a system setup like that, it would be unusably broken; no shell scripts would work. Pretty much all modern systems attempt to conform, at least superficially, to the POSIX standard, where the sh command processes the Shell Command Language specified in POSIX, which is roughly equivalent to the historical Bourne shell and which bash, dash, ash, etc. (shells which are commonly installed as /bin/sh) are all 99.9% compatible with.
You can completely ignore csh and similar. They're never installed as sh, and only folks who actually want to use them, or who get stuck using them as their interactive shell because some evil sysadmin setup the login shell defaults that way, ever have to care about them.
On any POSIX-like system, you can use
system("some_program > output.txt 2>&1");
This is because POSIX system is equivalent to calling sh, and POSIX sh supports this kind of redirection. This works independently of whether or not a user opening a terminal on the system will see a Csh prompt.
How do I redirect stderr (or stdout+stderr) to a file if I don't know which shell (bash, csh, dash) is interpreting my command?
You don't. Bourne-family shells and csh-family shells have different, incompatible syntax for redirecting stderr. In fact, csh and tcsh do not have a syntax to redirect only stderr at all -- they can redirect it only together with stdout.
If you really could be in any shell at all, then you're pretty much hosed with respect to doing much of anything. One could imagine an obscure, esoteric shell with completely incompatible syntax. For that matter, even an unusual configuration of a standard shell could trip you up -- for example if the IFS variable is set to an unusual value in a Bourne-family shell, then you'll have trouble executing any commands that don't take that into account.
If you can count on executing at least simple commands, then you could execute a known shell within the unknown one to process your command, but that oughtn't to be necessary for the case that seems to interest you.
Alternatively, is there a way to tell system() (or some other similar
C function?) to use /usr/bin/env bash instead of /bin/sh to interpret
the supplied command line?
Not on a POSIX-conforming system. POSIX specifies explicitly that the system() function executes the command by use of /bin/sh -c [the_command]. But this shouldn't be something to worry about, as /bin/sh should be a conforming POSIX shell, or at least pretty close to one. Definitely it should be a Bourne-family shell, which both bash and dash are, but tcsh most definitely is not.
The way to redirect the standard error stream in a POSIX shell is to use the 2> redirection operator (which is a special case of a more general redirection feature applicable to any file descriptor). Whatever shell /bin/sh actually is should recognize that syntax, and in particular bash and dash both do:
some_program 2> output.txt
I think, there is another possibility worth mentioning: You could open the file you want to redirect on stderr in your c-code prior to calling system(). You can dup() the original stderr first, and then restore it again.
fflush(stderr); // Flush pending output
int saved_stderr = dup(fileno(stderr));
int fd = open("output.txt", O_RDWR|O_CREAT|O_TRUNC, 0600);
dup2(fd, fileno(stderr));
close(fd);
system("some_program");
dup2(saved_stderr, fileno(stderr));
close(saved_stderr);
This should perform the output redirection as you need it.
If you don't know the shell.... of course you don't know how to redirect from it, despite of the fact that you can see what value the $SHELL has, and act in consequence:
char *shell = getenv("SHELL");
if (*shell) { /* no SHELL variable defined */
/* ... */
} else if (!strcmp(shell, "/bin/sh")) { /* bourne shell */
/* ... */
} /* ... more shells */
Despite of what you say in your question, it is quite unusual to rename /bin/sh to use another shell, as shell scripts use syntax that depends on that. The only case I know is with bash(1), and I have seen this only in Linux (and remarkably, last versions of solaris), but the syntax of bash(1) is a superset of the syntax of sh(1), making it possible to run shell scripts made for sh(1) with it. Renaming /bin/sh to perl for example, would make your system probably completely unusable, as many system tools depend of /bin/sh to be a bourne compatible shell.
By the way, the system(3) library function always calls sh(1) as the command interpreter, so there should be no problem to use it, but there's no solution to capture the output and process it by the parent process (indeed, the parent process is the sh(1) that system(3) fork(2)s)
Another thing you can do is to popen(3) a process. This call gives you a FILE pointer to a pipe of a process. You popen its input in case you popen(3) it for writing, and you popen its output if you want or read its output. Look at the manual for details, as I don't know now if it redirects only its standard output or it also redirects the standard error (I think only redirects standard output, for reasons discussed below, and only if you popen(3) it with a "r" flag).
FILE *f_in = popen("ps aux", "r");
/* read standard output of 'ps aux' command. */
pclose(f_in); /* closes the descriptor and waits for the child to finish */
Another thing you can do is to redirect yourself after fork(2)ing the child, and before the exec(2) call (this way you can decide if you want only stdout or if you want also stderr redirected back to you):
int fd[2];
int res = pipe(fd);
if (res < 0) {
perror("pipe");
exit(EXIT_FAILURE);
}
if ((res = fork()) < 0) {
perror("fork");
exit(EXIT_FAILURE);
} else if (res == 0) { /* child process */
dup2(fd[1], 1); /* redirect pipe to stdout */
dup2(fd[1], 2); /* redirect pipe also to stderr */
close(fd[1]); close(fd[0]); /* we don't need these */
execvp(program, argv);
perror("execvp");
exit(EXIT_FAILURE);
} else { /* parent process */
close(fd[1]); /* we are not going to write in the pipe */
FILE *f_in = fdopen(fd[0]);
/* read standard output and standard error from program from f_in FILE descriptor */
fclose(f_in);
wait(NULL); /* wait for child to finish */
}
You can see a complete example of this (not reading standard error, but it is easy to add --- you have only to add the second dup2() call from above) here. The program executes repeatedly a command you pass to it on the command line. It needs to get access to the output of the subprocess to count the lines, as between invocations, the program goes up as many lines as the program output, to make the next invocation to overlap the output of the last invocation. You can try it and play, making modifications as you like.
NOTE
In your sample redirection, when you use >&, you need to add a number after the ampersand, to indicate which descriptor you are dup()ing. As the number before the > is optional, the one after the & is mandatory. So, if you have not used it, prepare to receive an error (which probably you don't see if you are redirecting stderr) The idea of having two separate output descriptors is to allow you to redirect stdout and at the same time, conserve a channel where to put error messages.
It's possible to redirect everything that is written in the terminal to a process?
For example, after I started the process, if I write "command" in the terminal, this should be redirected to a pipe from my process or something like this.
Yes, it should be practical to redirect all terminal output from your program (and all of its child processes) after your program has started. Unix programs usually write to the terminal by writing to standard output (stdout). Standard output is always on the file descriptor number 1 (the C constant is STDOUT_FILENO), for all processes. You can use the dup2() system call to replace any file descriptor number with another file descriptor.
So you can e.g. create a pipe using int fds[2]; pipe(fds);. Then fds[1] will be a file descriptor number that you can use to write to the pipe. If you do dup2(fds[1], STDOUT_FILENO); then standard output will also write to the pipe. (You can close(fds[1]); afterwards since you probably don't need it, now that you can use stdout instead.)
You can also open a file for writing with fd = open("filename", O_WRONLY); and then dup2(fd, STDOUT_FILENO); so everything written to stdout goes into your file.
Note that you need to redirect stdout at the very beginning of your program before doing anything that might write to stdout.
The above trick will make standard output go to your pipe instead of the terminal. If you want the output to go to the terminal, and also get a copy of the output in a pipe of file, that's more difficult but can also be done. You need to create an internal pipe, then dup2(that_pipe, STDOUT_FILENO); so stdout writes to that pipe. Then you need to read from that pipe (probably using poll() then read()) and write everything you got to both 1) the terminal and 2) to another pipe or file that is going outside your program. So you need two pipes if you want to copy output.
The tee command does this (copy stdout to files) from the shell.
This dup2() approach is not bulletproof because a Unix terminal (even when using a GUI terminal emulator instead of a hardware console) is a device in /dev. You can type tty in a shell or use ttyname(STDOUT_FILENO) in C to see which file in /dev corresponds to the terminal that stdout is writing to. In principle, any program (under the same user account) could open the terminal device using that filename and write to it without asking for permission from any other program. You can easily try this from the shell using the write program:
echo hello world | write $(whoami) /dev/ttys123
where /dev/ttys123 is whatever you got by typing tty in some other terminal window (the name looks a bit different on different operating systems, e.g. Linux and MacOS). You should see hello world appear in that other window.
From a child process, no. You must set this up in the parent preocess, and have it propagate downwards to children (barring some kind of crazy hack).
From the shell, you can redirect.
exec >file
This will redirect standard output to file, and it will apply to all future commands run in the shell. You can make this into a function, if you like.
I have a Linux process running in the background. I want to take over its stdin/out/err over SSH and also be the terminal controller. The "original" file descriptors are pseudo terminals, too.
I have tried Reptyr and dupx. Reptyr fails around vfork, but dupx works very well. The GDB script it generated:
attach 123
set $fd=open("/dev/pts/14", 0)
set $xd=dup(0)
call dup2($fd, 0)
call close($fd)
call close($xd)
set $fd=open("/dev/pts/14", 1089)
set $xd=dup(1)
call dup2($fd, 1)
call close($fd)
call write($xd, "Remaining standard output of 123 is redirected to /dev/pts/14\n", 62)
call close($xd)
set $fd=open("/dev/pts/14", 1089)
set $xd=dup(2)
call dup2($fd, 2)
call close($fd)
call write($xd, "Remaining standard error of 123 is redircted to /dev/pts/14\n", 60)
call close($xd)
As soon as the dupx command finished, the shell is not returned and the target app receives my input (via pts/14) immediately.
Now I want to achieve the same result using my standalone binary application. I've ported the same syscalls (dup/dup2/close, etc) what being executed by the gdb by script driven by dupx:
int fd; int xd;
char* s = "Remaining standard output is redirected to new terminal\n";
fd = open(argv[1], O_RDONLY);
xd = dup( STDIN_FILENO);
dup2(fd, STDIN_FILENO );
close(fd);
close(xd);
fd = open(argv[1], O_WRONLY|O_CREAT|O_APPEND);
xd = dup( STDOUT_FILENO);
dup2(fd, STDOUT_FILENO);
close(fd);
write(xd, s, strlen(s));
close(xd);
fd = open(argv[1], O_WRONLY|O_CREAT|O_APPEND);
xd = dup( STDERR_FILENO);
dup2(fd, STDERR_FILENO);
close(fd);
write(xd, s, strlen(s));
close(xd);
Running the snipplet above is done by injecting a shared library into the remote process via sigstop/ptrace attach/dlopen/etc (using a tool similar to hotpatch). Lets consider this part of the problem to be safe and working reliable: after doing all this, the file descriptors of the target process are changed as I wanted. I can verify it by simply checking /proc/pidof target/fd.
However, the shell returns and it still receives all my input, not the target app.
I noticed if I simply attach/detach with gdb after this point (= fds changed by the injected C code) without actually changing anything, the desired behavior is accomplished (mean: the shell is not returned but the target app starts receiving my input). The command is:
gdb --pid=`pidof target` --batch --ex=quit
And now my question is: how? What happens in the background? How can I do the same without gdb? I've tried stracing gdb to get some hints, and also tried playing with the tty ioctl API's without any luck.
Please note, that obtaining the terminal controller status (if that is the key of this problem at all) by the fork/setsid way what Reptyr uses is not acceptable for me: I want to avoid forking.
Additionally, I cant control starting the target, so "why don't you run it in screen" is no answer here.
I've ssh access, thats where pts/14 was coming from. Shell and the
target app might be competing, but I've never experienced such
behaviour; dupx alwaysed did what I wanted in this scenario.
Well, sitting and wondering why the known problem by chance didn't show up in the past won't solve it, even if this point would be clarified. The way to go is to make it work by design rather than by accident. For this purpose it is necessary for your standalone binary application to not return to the shell (to avoid the concurrent reading of input) while the input is supposed to go to the target app.
See e. g. also Redirect input from one terminal to another, Why does tapping a TTY device only capture every other character?
I am trying to create a function which takes a shell command as an argument , uses fork to spawn a new process which executes the command. I also want to redirect the standard output of the command so the caller of the function can read it using a FILE* pointer.
static FILE* runCommand(char* command){
int pfd[2];
if(pipe(pfd)<0)
return NULL;
if(pid=fork()==0){ //child
close(pfd[0]);
dup2(pfd[1],1); //redirect output to pipe for writing
execlp(command,(char*)0);
}
close(pfd[1]);
//return a file pointer/descriptor here?
}
I am not sure how to return a file pointer which can be used to read the output of the command. Also is that the correct way to execute a command on the shell?
ps. I read about popen but there is a good reason I can't use it, thus I have to implement this functionality myself.
Thank you
One bug in that code is that you assign to a variable pid that is not declared anywhere. And pid will always be 1 in the parent, because the code as written is equivalent to pid=(fork()==0) rather than (pid=fork())==0.
You should also close pfd[1] after the dup2 call. And for good measure, check for errors from dup2 and execlp.
The answer to your real question is to use fdopen.
Use fdopen to associate an existing file descriptor with a FILE * object. Everything else looks pretty good.
I want to execute a command using system() command or execl and want to capture the output directly in a buffer in C. Is ther any possibility to capture the output in a buffer using dup() system call or using pipe(). I dont want to use any file in between using mkstemp or any other temporary file. please help me in this.Thanks in advance.
I tried it with fork() creating two process and piping the output and it is working.However I dont want to use fork system call since i am going to run the module infinitely using seperate thread and it is invoking lot of fork() and system is running out of resources sometimes after.
To be clear about what i am doing is capturing an output of a shell script in a buffer processing the ouput and displaying it in a window which i have designed using ncurses.Thankyou.
Here is some code for capturing the output of program; it uses exec() instead of system(), but that is straightforward to accomodate by invoking the shell directly:
How can I implement 'tee' programmatically in C?
void tee(const char* fname) {
int pipe_fd[2];
check(pipe(pipe_fd));
const pid_t pid = fork();
check(pid);
if(!pid) { // our log child
close(pipe_fd[1]); // Close unused write end
FILE* logFile = fname? fopen(fname,"a"): NULL;
if(fname && !logFile)
fprintf(stderr,"cannot open log file \"%s\": %d (%s)\n",fname,errno,strerror(errno));
char ch;
while(read(pipe_fd[0],&ch,1) > 0) {
//### any timestamp logic or whatever here
putchar(ch);
if(logFile)
fputc(ch,logFile);
if('\n'==ch) {
fflush(stdout);
if(logFile)
fflush(logFile);
}
}
putchar('\n');
close(pipe_fd[0]);
if(logFile)
fclose(logFile);
exit(EXIT_SUCCESS);
} else {
close(pipe_fd[0]); // Close unused read end
// redirect stdout and stderr
dup2(pipe_fd[1],STDOUT_FILENO);
dup2(pipe_fd[1],STDERR_FILENO);
close(pipe_fd[1]);
}
}
A simple way is to use popen ( http://www.opengroup.org/onlinepubs/007908799/xsh/popen.html), which returns a FILE*.
You can try popen(), but your fundamental problem is running too many processes. You have to make sure your commands finish, otherwise you will end up with exactly the problems you're having. popen() internally calls fork() anyway (or the effect is as if it did).
So, in the end, you have to make sure that the program you want to run from your threads exits "soon enough".
You want to use a sequence like this:
Call pipe once per stream you want to create (eg. stdin, stdout, stderr)
Call fork
in the child
close the parent end of the handles
close any other handles you have open
set up stdin, stdout, stderr to be the appropriate child side of the pipe
exec your desired command
If that fails, die.
in the parent
close the child side of the handles
Read and write to the pipes as appropriate
When done, call waitpid() (or similar) to clean up the child process.
Beware of blocking and buffering. You don't want your parent process to block on a write while the child is blocked on a read; make sure you use non-blocking I/O or threads to deal with those issues.
If you are have implemented a C program and you want to execute a script, you want to use a fork(). Unless you are willing to consider embedding the script interpreter in your program, you have to use fork() (system() uses fork() internally).
If you are running out of resources, most likely, you are not reaping your children. Until the parent process get the exit code, the OS needs keeps the child around as a 'zombie' process. You need to issue a wait() call to get the OS to free up the final resources associated with the child.