procfs.c
for(i=0;i<10;i++)
{
//linux command to check process status
sprintf(cmd, "cat /proc/%d/status", pid[i]);
pf = popen(cmd,"r");
fread(data, 1024, 1, pf);
pclose(pf);
//......big chunk of code afterwards
}
This is part of the code I'm running on my ubuntu. Basically, pid array has some the process id's, and I want those data to be parsed in some sort of way - which did succeed, so this isn't the problem.
The problem is with some part of the structure. Initially when I saved the pid array, I used "ls /proc/" command - the same way I used "cat /proc/%d/status" command in the above code - to check the /proc/ folder for all the processes that are currently running. The above code runs some time later, so when I use the pid array list for execution, some programs are no longer running, and thus, is not in /proc/ folder (for example, the program itself). So while all the pid data are printed out the way I want them to, some data come out as below:
In order to cope with this I added a line of code like this:
if(!pf) continue;
I thought that this would see that the command has failed, and skip this iteration, but it didn't change anything.
Is there any way to deal with that error message?
edit: I also tried if(pf < 0), but this didn't work either.
Use the stat function to see if a file exists, which works perfectly well for /proc files.
#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
struct stat buf;
int ret = stat("/proc/2023", &buf);
if(ret == -1) {
//file doesn't exist, so proc with pid 2023 isn't running
}
else {
//file exists, so proc with pid 2023 is running
}
Incorporating this into your loop, we have:
for(i=0;i<10;i++)
{
struct stat buf;
sprintf(cmd, "/proc/%d", pid[i]);
int ret = stat(cmd, &buf);
if(ret == -1) continue;
//linux command to check process status
sprintf(cmd, "cat /proc/%d/status", pid[i]);
pf = popen(cmd,"r");
fread(data, 1024, 1, pf);
pclose(pf);
//......big chunk of code afterwards
}
Related
I hope to see the output of the command executed by popen as soon as possible.So I change the buffering type of the file stream returned by popen to line buffered. As per the document, setvbuf seems work for this goal. I did a simple test on Ubuntu16.4, it does not make any difference indeed.
Here is the code snippet which I used to do the said test:
#include <stdio.h>
#include <string.h>
int main(void)
{
char buffer[1024];
memset(buffer, 0, 1024);
FILE* file = popen("bash -c \"for i in 1 2 3 4 5;do echo -e -n 'thanks a lot\n'; sleep 1; done\" ", "r");
if(NULL != file)
{
int size;
printf("setvbuf returns %d\n", setvbuf(file, NULL, _IOLBF, 0));
while((size = fread(buffer, 1, 1024, file))>0)
{
printf("size=%d\n", size);
memset(buffer, 0, 1024);
}
}
return 0;
}
Here is the output of the code snippet which runs on Ubuntu16.04:
setvbuf returns 0
//about five seconds later
size=65
As per the document, which says that:
The function setvbuf() returns 0 on success.
As per the output above, setvbuf(file, NULL, _IOLBF, 0) has successfully set the buffing type of file returned by popen to line buffered.But the output of the aforementioned code snippet indicates it still uses the default block buffered.
But when I tried getline, it could achieve the goal, which is really out of my expectation.
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
int main(void)
{
FILE* file = popen("bash -c \"for i in 1 2 3 4 5;do echo -e -n 'thanks a lot\n'; sleep 1; done\" ", "r");
char* line=NULL;
size_t len;
if(NULL != file)
{
// std::cout << setvbuf(file, NULL, _IOLBF, 0) << std::endl; //setvbuf has not been called
while(getline(&line, &len, file)>0)
{
printf("strlen(line)=%lu\n", strlen(line));
free(line);
line = NULL;
}
}
free(line);
return 0;
}
Here is the output:
//one second later
strlen(line)=13
//one second later
strlen(line)=13
//one second later
strlen(line)=13
//one second later
strlen(line)=13
//one second later
strlen(line)=13
I am really conscious about why getline could acquire the output of the pipe as soon as possible, whereas setvbuf & fread does not work.
getline stops reading once it gets to a newline. fread keeps reading until it reads as much data as you specified (in your case, 1024 bytes) or it encounters EOF or an error. This has nothing to do with buffering. You might want to look at read to see if it's closer to what you want.
#Jeremy Friesner'answer:
In my experience, the call to setvbuf(stdout, NULL, _IONBF, 0) needs to be executed inside the child process in order to get the behavior you want. (Of course that's much easier to achive if the child process's executable is one you control rather than /bin/bash). popen() in the parent process can't give you data that the child process hasn't supplied to it yet.
I had this simple shell like program that works both in interactive and non-interactive mode. I have simplified the code as much as I can to present my question, but it is still a bit long, so sorry for that!
#include <stdio.h>
#include <stdlib.h>
#include <sys/wait.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
/**
*main-entry point for gbk
*Return: returns the index of 0 on sucess
*/
int main(void)
{
char *cmd = malloc(1 * sizeof(char)), *cmdargs[2];
size_t cmdlen = 0;
int childid, len;
struct stat cmdinfo;
while (1)
{
printf("#cisfun$ ");
len = getline(&cmd, &cmdlen, stdin);
if (len == -1)
{
free(cmd);
exit(-1);
}
/*replace the ending new line with \0*/
cmd[len - 1] = '\0';
cmdargs[0] = cmd;
cmdargs[1] = NULL;
childid = fork();
if (childid == 0)
{
if (stat(*cmdargs, &cmdinfo) == 0 && cmdinfo.st_mode & S_IXUSR)
execve(cmdargs[0], cmdargs, NULL);
else
printf("%s: command not found\n", *cmdargs);
exit(0);
}
else
wait(NULL);
}
free(cmd);
exit(EXIT_SUCCESS);
}
To summarize what this program does, it will first print the prompt #cisfun$ , waits for an input in interactive mode and takes the piped value in non-interactive mode, creates a child process, the child process checks if the string passed is a valid executable binary, and if it is, it executes it other wise it prints a command not found message and prompts again.
I have got this program to work fine for most of the scenarios in interactive mode, but when I run it in non-interactive mode all sorts of crazy (unexpected) things start to happen.
For example, when I run echo "/bin/ls"|./a.out, (a.out is the name of the compiled program)
you would first expect the #cisfun$ message to be printed since that is the first thing performed in the while loop, and then the output of the /bin/ls command, and finally #cisfun$ prompt, but that isn't what actually happens. Here is what happens,
It is very weird the ls command is run even before the first print message. I, at first, thought there was some threading going on and the printf was slower than the child process executing the ls command. But I am not sure if that is true as I am a noob. and also things get a bit crazier if I was printing a message with '\n' at the end rather than just a string. (if I change printf("#cisfun$ "); to printf("#cisfun$\n");) the following happens,
It works as it should, so it got me thinking what is the relation between '\n', fork and speed of printf. Just in short what is the explanation for this.
The second question I have is, why doesn't my program execute the first command and go to an interactive mode, I don't understand why it terminates after printing the second #cisfun$ message. By checking the status code (255) after exit I have realized that the effect is the same as pressing ctr+D in the interactive mode, which I believe is exited by the getline function. But I dont understand why EOF is being inserted in the second prompt.
I've found that a open filestream will get messed up if we do fork() before closing it. It is well known that concurrency, i.e., race conditions can happen when parent and child process want to modify the filestream. However, even when the child process doesn't ever touch the filestream, it still has undefined behavior. I was wondering if someone can explain this maybe from how the kernel deals with a filestream during the stages where child process is forked and exited.
Below is a quick snippet of a strange behavior:
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/wait.h>
int main() {
// Open file
FILE* fp = fopen("test.txt", "r");
int count = 0;
char* buffer = NULL;
size_t capacity = 0;
ssize_t line = 0;
while ( (line = getline(&buffer, &capacity, fp)) != -1 ) {
if (line > 0 && buffer[line - 1] == '\n') // remove the end '\n'
buffer[line - 1] = 0;
pid_t pid = fork();
if (pid == 0) {
// fclose(fp); // Magic line here: when you add this, everything is fine
if (*buffer == '2')
execlp("xyz", "xyz", NULL);
else
execlp("pwd", "pwd", NULL);
exit(1);
} else {
waitpid(pid, NULL, 0);
}
count++;
}
printf("Loops: %d\n", count);
return 0;
}
Just copy the code into a new file (e.g., test.c). And create a .txt file test.txt with the simple content
1
2
3
4
and run
$ gcc test.c && ./a.out
There are 4 lines in the file. The loops is expected to read each line and execute exactly 4 times (1 2 3 4). And I choose to let it exec an invalid command "xyz" when it's in the 2nd loop. Then, you will find the loop actually executes 6 times (1 2 3 4 3 4)! The fact is that, when all four commands executed are valid, nothing will go wrong. But if there is an invalid command executed, every command after it will be executed twice. (Please note that this strange behavior only occurs with Linux machine, my Mac OS is doing okay, not sure about Windows. So the problem is platform-dependent?)
It looks like whenever I fork(), the filestream in parent is no longer promised to be the old fp (non-deterministic behavior), even when my child process doesn't touch it.
A temporary solution I found is: fclose(fp) in child process. This will silence the above strange behavior, but in more complex conditions, there are still other things can be observed. It would be appreciated if somebody can give me some insight into this problem. Thanks
As said in the comments already you need to close open file descriptors before calling exec.
In this blogpost (section 4) there is a neat code sample you can use to ensure all fds are closed even in complex applications where you don't always know what files are open at the moment:
for ( i=getdtablesize(); i>2; --i)
close(i); /* close all descriptors */
(slightly modified to keep stdin, stdout, stderr open)
It's kind of hacky but it works. If you want to avoid that you can also set the O_CLOEXEC flag on each file descriptor that you open. Since when using fopen you do not directly call open() you can accomplish this by adding the 'e' flag to it (when using glibc >= 2.7):
FILE* fp = fopen("test.txt", "er");
When calling exec*() all file descriptors with this flag are automatically closed.
I have two file open in two different processes. There's a pipe connecting the two. Is it possible to write directly from one file to another? Especially if the process reading doesn't know the size of the file it's trying to read?
I was hoping to do something like this
#define length 100
int main(){
int frk = fork();
int pip[2];
pipe(pip);
if (frk==0){ //child
FILE* fp fopen("file1", "r");
write(pip[1],fp,length);
}
else {
FILE* fp fopen("file2", "w");
read(pip[0],fp,length);
}
Is it possible to write directly from one file to another?
C does not provide any mechanism for that, and it seems like it would require specialized hardware support. The standard I/O paradigm is that data get read from their source into memory or written from memory to their destination. That pesky "memory" in the middle means copying from one file to another cannot be direct.
Of course, you can write a function or program that performs such a copy, hiding the details from you. This is what the cp command does, after all, but the C standard library does not contain a function for that purpose.
Especially if the process reading doesn't know the size of the file it's trying to read?
That bit isn't very important. One simply reads and then writes (only) what one has read, repeating until there is nothing more to read. "Nothing more to read" means that a read attempt indicates by its return value that the end of the file has been reached.
If you want one process to read one file and the other to write that data to another file, using a pipe to convey data between the two, then you need both processes to implement that pattern. One reads from the source file and writes to the pipe, and the other reads from the pipe and writes to the destination file.
Special note: for the process reading from the pipe to detect EOF on that pipe, the other end has to be closed, in both processes. After the fork, each process can and should close the pipe end that it doesn't intend to use. The one using the write end then closes that end when it has nothing more to write to it.
In other unix systems, like BSD, there's a call to connect directly two file descriptors to do what you want, but don't know if there's a system call to do that in linux. Anywya, this cannot be done with FILE * descriptors, as these are the instance of a buffered file used by <stdio.h> library to represent a file. You can get the file descriptor (as the system knows it) of a FILE * instance by a call to the getfd(3) function call.
The semantics you are trying to get from the system are quite elaborate, as you want something to pass directly the data from one file descriptor to another, without intervention of any process (directly in the kernel), and the kernel needs for that a pool of threads to do the work of copying directly from the read calls to the write ones.
The old way of doing this is to create a thread that makes the work of reading from one file descriptor (not a FILE * pointer) and write to the other.
Another thing to comment is that the pipe(2) system call gives you two connected descriptors, that allow you to read(2) in one (the 0 index) what is write(2)n in the second (the 1 index). If you fork(2) a second process, and you do the pipe(2) call on both, you will have two pipes (with two descriptors each), one in each process, with no relationship between them. You will be able only to communicate each process with itself, but not with the other (which doesn't know anything about the other process' pipe descriptors) so no communication between them will be possible.
Next is a complete example of what you try to do:
#include <errno.h>
#include <stdlib.h>
#include <string.h>
#include <sys/wait.h>
#include <fcntl.h>
#include <stdio.h>
#include <sys/types.h>
#include <unistd.h>
#define length 100
#define FMT(fmt) "pid=%d:"__FILE__":%d:%s: " fmt, getpid(), __LINE__, __func__
#define ERR(fmt, ...) do { \
fprintf(stderr, \
FMT(fmt ": %s (errno = %d)\n"), \
##__VA_ARGS__, \
strerror(errno), errno); \
exit(1); \
} while(0)
void copy(int fdi, int fdo)
{
unsigned char buffer[length];
ssize_t res, nread;
while((nread = res = read(fdi, buffer, sizeof buffer)) > 0) {
res = write(fdo, buffer, nread);
if (res < 0) ERR("write");
} /* while */
if (res < 0) ERR("read");
} /* copy */
int main()
{
int pip[2];
int res;
res = pipe(pip);
if (res < 0) ERR("pipe");
char *filename;
switch (res = fork()) {
case -1: /* error */
ERR("fork");
case 0: /* child */
filename = "file1";
res = open(filename, O_RDONLY);
if (res < 0) ERR("open \"%s\"", filename);
close(pip[0]);
copy(res, pip[1]);
break;
default: /* parent, we got the child's pid in res */
filename = "file2";
res = open(filename, O_CREAT | O_TRUNC | O_WRONLY, 0666);
if (res < 0) ERR("open \"%s\"", filename);
close(pip[1]);
copy(pip[0], res);
int status;
res = wait(&status); /* wait for the child to finish */
if (res < 0) ERR("wait");
fprintf(stderr,
FMT("The child %d finished with exit code %d\n"),
res,
status);
break;
} /* switch */
exit(0);
} /* main */
I need to load a *.ml file into the Ocaml toplevel (the interactive interpreter, when you type 'ocaml' in a shell) and then send an instruction from a Matlab process, get back the result of the instruction, send back another instruction, ...
I've made this C program. The parent process gets the Matlab's instruction from a named pipe, sends it to the child process (with ocaml running) and gets the response back so it can send it to Matlab.
But there is some kind of bug: when I send an instruction, I get back some weird characters, I send another instruction and then I receive the response of the first instruction...
(I didn't copy the perror() test to have less text)
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
int main(void) {
// Parent -> Child
int pipe_in[2];
// Child -> parent
int pipe_out[2];
/*
pipe[0] = output
pipe[1] = input
*/
pipe(pipe_in);
pipe(pipe_out);
pid_t pid;
if ((pid = fork()) == 0) {
// CHILD SIDE
close(pipe_in[1]);
close(pipe_out[0]);
dup2(pipe_in[0], STDIN_FILENO);
dup2(pipe_out[1], STDOUT_FILENO);
dup2(pipe_out[1], STDERR_FILENO);
close(pipe_in[0]);
close(pipe_out[1]);
char *args[] = {"ocaml", NULL};
execvp("ocaml", args);
printf("FAIL\n");
exit(EXIT_FAILURE);
} else {
// PARENT SIDE
printf("[*] PID : %d\n", (int) pid);
close(pipe_in[0]);
close(pipe_out[1]);
char cmd[1024];
char feedback[1024];
ssize_t cmd_read;
ssize_t feedback_read = sizeof(feedback);
while (1) {
// Get the instruction from Matlab.
printf("[>] ");
int fifo_in = open("/tmp/pipe_in", O_RDONLY);
cmd_read = read(fifo_in, cmd, sizeof(cmd));
close(fifo_in);
printf("%s\n", cmd);
// Send the instruction to the ocaml interpreter.
write(pipe_in[1], cmd, cmd_read);
// Read the response of the ocaml interpreter.
while (feedback_read == sizeof(feedback)) {
feedback_read = read(pipe_out[0], feedback, sizeof(feedback));
printf("[-] %d\n", (int) feedback_read);
}
printf("[<] %s\n", feedback);
// Send to Matlab the response.
int fifo_out = open("/tmp/pipe_out", O_WRONLY);
write(fifo_out, feedback, feedback_read);
close(fifo_out);
cmd_read = 0;
feedback_read = sizeof(feedback);
}
close(pipe_in[1]);
close(pipe_out[0]);
}
}
I compile the code with gcc -Wall -std=c99 -o tphr tphr.c
I run the programm in one shell and in another :
> printf 'let x = 10;;\n' > /tmp/pipe_in
> cat /tmp/pipe_out
OCaml version 4.03.0
# %
> printf 'let y = 5;;\n' > /tmp/pipe_in
> cat /tmp/pipe_out
val x : int = 10
# %
How can I fix the result ?
If, by "weird characters", you mean
OCaml version 4.03.0
that is simply because this is what the OCaml toplevel prints out on startup. So you need to read this line when your own program starts up.
If you mean the # symbol, also known as the prompt, you can turn it off by running ocaml -nopromt.
You don't want to run the interactive ocaml toplevel here for multiple reasons:
the toplevel parses the configfiles of the user and then loads different modules. This can change the available values, change behaviour, and make your matlab process get different results per user.
the output of the toplevel may change between versions making it difficult to parse and return the right reply to matlab
Did you know that you can call the toplevel from ocaml bytecode to interprete strings? I suggest dumping the C code and writing ocaml byte code to read from the pipe, interpret the command and reply with the result.