I am using tcpstat in a linux environment. I want to capture its output in a C program even though it has not finished. I tried using the popen() function, but it can only process the output after the program has finished. I want to process the output of tcpstat on the fly as and when it prints it on standard output. How do i do so?
For example,
$ tcpstat -i wlan0 1
Time:1297790227 n=2 avg=102.50 stddev=42.50 bps=1640.00
Time:1297790228 n=11 avg=86.36 stddev=19.05 bps=7600.00
Time:1297790229 n=32 avg=607.97 stddev=635.89 bps=155640.00
Time:1297790230 n=13 avg=582.92 stddev=585.55 bps=60624.00
The above output keeps going on till infinity. So I want to process the output in a C program as and when tcpstat outputs something onto stdout.
Thanks and Regards,
Hrishikesh Murali
Run tcpstat -i wlan0 -a 1 | your_program and read from the standard input in your program. This way the shell will take care of the piping.
The popen library function and the pipe system call can be used to achieve the same result at a lower level. You may want to take a look at named pipes too - they appear like files in userspace and can be manipulated in the same way.
Run tcpstat with the -F option, this will cause it to flush its output on every interval. (instead of using the default block buffering for stdout)
In addition, you may want to explicitly disable the buffering on your popen FILE handle using setbuf, eg.
setbuf(popen_fd, NULL);
Alternately, you can set it to be line buffered, using setlinebuf
setlinebuf(popen_fd);
Related
I'm now working on a small C program in Linux. Let me explain you what I want to do with a sample Linux command below
ls | grep hello
The above command is executed in the below passion (Let me know if I've got this wrong)
ls command will be executed first
Output will be given to grep command which will again generate output by matching "hello"
Now I would like to write a C program which takes the piped output of one command as input. Means, In the similar passion of how "grep" program was able to get the input from ls command (in my example above).
Similar question has been asked by another user here, but for some reason this thread has been marked as "Not a valid question"
I initially thought we can get this as a command line argument to C program. But this is not the case.
If you pipe the output from one command into another, that output will be available on the receiving process's standard input (stdin).
You can access it using the usual scanf or fread functions. scanf and the like operate on stdin by default (in the same way that printf operates on stdout by default; in the absence of a pipe, stdin is attached to the terminal), and the C standard library provides a FILE *stdin for functions like fread that read from a FILE stream.
POSIX also provides a STDIN_FILENO macro in unistd.h, for functions that operate one file descriptors instead. This will essentially always be 0, but it's bad form to rely on that being the case.
If fact, ls and grep starts at the same time.
ls | grep hello means, use ls's standard output as grep's standard input. ls write results to standard output, grep waits and reads any output from standard input at once.
Still have doubts? Do an experiment. run
find / | grep usr
find / will list all files on the computer, it should take a lot of time.
If ls runs first, then OS gives the output to grep, we should wait a long time with blank screen until find finished and grep started. But, we can see the results at once, that's a proof for that.
I am working in OSX and using bash for my shell. I have a script which calls an executable hundreds of times, and each call is independent of the other. Therefore I am going to run this code in parallel. However, each call to the executable appends output to a community text file on a new line.
The ordering of the text file is not of importance (although it would be nice, but totally not worth over complicating since I can just use unix sort command), but what is, is that every call of the executable properly printed to the file. My concern is that if I run the script in parallel that the by some freak accident, two threads will check out the text file, print to it and then save different copies back to the original directory of the text file. Thus nullifying one of the writes to the file.
Does this actually happen, or is my understanding of printing to a file flawed? I don't fully know if this would also be a case by case bases so I will provide some mock code of what is being done in my program below.
Script:
#!/bin/sh
abs=$1
input=$(echo "$abs" | awk '{print 0.004 + 0.005*$1 }')
./program input
"./program":
~~Normal .c file stuff here~~
~~VALUE magically calculated here~~
~~run number is pulled out of input and assigned to index for sorting~~
FILE *fpp;
fpp = fopen("Doc.txt","a");
fprintf(fpp,"%d, %.3f\n", index, VALUE);
fclose(fpp);
~Closing events of program.c~~
Commands to run script in parallel in bash:
printf "%s\n" {0..199} | xargs -P 8 -n 1 ./program
Thanks for any help you guys can offer.
A write() call (like fwrite()) with the append flag set in open() (like during fopen()) is guaranteed to avoid the race condition you describe.
O_APPEND
If set, the file offset shall be set to the end of the file prior to each write.
From: POSIX specifications for open:
opengroup.org open
Race conditions are what you are thinking of.
Not 100% sure but if you simple append to the end of the file rather than opening it and editing it should be right
If you have the option, make your program write to standard output instead of directly to a file. Then you can let the shell merge the output of your programs:
printf "%s\n" {0..199} | parallel -P 8 -n 1 ./program > merged_output.txt
Yeah, that looks like a recipe for disaster. If those processes both hit opening the file at the roughly the same time, only one will "take".
I suggest either (easier) writing to separate files then catting them together when the processing is done, or (harder) sending all results to a consumer process that will write the file for everyone.
I have a C program which gives some output. I am compiling the C program via the shell but I need the output from the run C program and store in shell.
Edit.
Save the output to a shell Variable.
I assume that you want to store the output of the program in a variable. Unix shells offer a facility that's called command substitution to do just that. Depending on your shell, you can do either :
output=$(./run)
or
output=`./run`
Bash supports both. If, however, you want to save the output to a file, you will need to redirect the standard output stream to a file. You can do this like this :
./run > output.txt
Or, if you want to see the output while the program is running and save it to an output file as well, you can use the tee utility and pipe the output of your program to it.
./run | tee output.txt
You can redirect your output into a file like this:
./run > file
If you want to store it into a variable, you have to decide which shell we're talking about. It depends whether you have a windows shell, or linux bash..
Hello I am currently trying to printf a certain part of the command line when i run my program. I currently run my program as follows
zcat gcc | sim
What i am trying to accomplish is print out to a seperate file the 'gcc' part of the command line. Is there any way to accomplish this. Thanks for the help
I forgot to mention this is in c not C++
The 'sim' program only knows that it has data from stdin. It doesn't know what is handing the data to stdin - it may be another program, or redirected input, or from a device.
From 'sim' POV, there's only data from stdin.
IF you can guarantee that you will always receive data from another process via a pipe, you can start looking for process headers matching the 'sim' process, and from then extracting process info from associated processes. However, I suspect that this is more than you're willing to go.
You are passing output of zcat (through pipe) to your program, it doesn't know details of arguments passed to zcat.
You can explicitly pass it as a part of input stream though,
(echo 'zcat gcc'; zcat gcc ) | sim
but that is a hack. It may be Ok if you are planning to run your
program this way only.
There are several ways of redirecting the output of a child process:
using freopen(3)
using dup(3)
using popen(3)
...
What should one pick if all is wanted is to execute a child process and have it output saved in a given file, pretty much like the ls > files.txt works?
What is normally used by shells?
You can discover what your favorite shell uses by strace(1)ing your shell.
In one terminal:
echo $$
In another terminal:
strace -o /tmp/shell -f -p [PID from the first shell]
In the first terminal again:
ls > files.txt
In the second terminal, ^C your strace(1) command and then edit the /tmp/shell output file to see what system calls it made to do the redirection.
freopen(3) manipulates the C standard IO FILE* pointers. All this will be thrown away on the other side of the execve(2) call, because it is maintained in user memory. You could use this after the execve(2) call, but that would be awkward to use generically.
popen(3) opens a single unidirectional pipe(7). This is useful, but extremely limited -- you get either the standard output descriptor or the standard input descriptor. This would fail for something like ls | grep foo | sort where both input and output must be redirected. So this is a poor choice.
dup2(2) will manage file descriptors -- a kernel-implemented resource -- so it will persist across execve(2) calls and you can set up as many file descriptors as you need, which is nice for ls > /tmp/output 2> /tmp/error or handling both input and output: ls | sort | uniq.
There is another mechanism: pty(7) handling. The forkpty(3), openpty(3), functions can manage a new pseudo-terminal device created specifically to handle another program. The Advanced Programming in the Unix Environment, 2nd edition book has a very nice pty example program in its source code, though if you're having trouble understanding why this would be useful, take a look at the script(1) program -- it creates a new pseudo-terminal and uses it to record all input and output to and from programs and stores the transcript to a file for later playback or documentation. You can also use it to script actions in interactive programs, similar to expect(1).
I would expect to find dup2() used mainly.
Neither popen() nor freopen() is designed to handle redirections such as 3>&7. Up to a point, dup() could be used, but the 3>&7 example shows where dup() starts to creak; you'd have to ensure that file descriptors 4, 5, and 6 are open (and 7 is not) before it would handle what dup2() would do without fuss.