Below command will record the data from default device and output it on stdout and aplay will play the data from stdin.
arecord -D hw:0 | aplay -D hw:1 -
Why we prefer stdin and stdout instead of writing into a file and reading from it as below?
arecord -D hw:0 test.wav | aplay -D hw:1 test.wav
Using a pipe for this operation is more efficient and effective than using a file, simply because of the following reasons:
1) Pipe (|) is an interprocess communication technique. The output of one process is directly sent to the input of another process using a kernel-based buffer. So this gives faster speed than writing something to a file in hard disk and reading it from it. This depends however on other factors also. Generally, the kernel also writes and reads small files from buffers and disk caches.
2) Using interprocess communication technique also helps in getting concurrent operation. Instead, if you had to write something to a file and then read from it, it would have to be performed in steps, so you would have lost the concurrency.
I assume you meant
arecord -D hw:0 test.wav && aplay -D hw:1 test.wav
instead of
arecord -D hw:0 test.wav | aplay -D hw:1 test.wav
Related
I would like to read files fast from my disk, I have tried three methods:
bash: cat file_path > /dev/null
d: auto data = read(file_path);
bash: rsync -av -P file_path path_to_fast_file_system
Method 1. reads at 470MB/s, which is ok speed wise, but the files end up in /dev/null which is not suitable for my needs.
Method 2. and 3. read at 140MB/s which is slow, but they do what I need.
Why is it three times slower to read into memory than to read into /dev/null?
Please let me know how I can read the data of a file at the same speed as cat to /dev/null but into memory or through rsync?
I'm trying to capture the standard output from tshark through a program in C.
For that, I use popen() call to open tshark process and read from the returned FILE stream.
Code sample:
#include <stdio.h>
#include <stdlib.h>
int main() {
FILE* pipe_fd = popen("tshark -i eth0 -R icmp -2 -T fields -e icmp.checksum -e icmp.seq", "r");
//FILE* pipe_fd = popen("lsof", "r");
if (!pipe_fd) {
fprintf(stderr, "popen failed.\n");
return EXIT_FAILURE;
}
char buffer[2048];
while (NULL != fgets(buffer, sizeof(buffer), pipe_fd)) {
fprintf(stdout, "SO: %s", buffer);
}
pclose(pipe_fd);
printf("tdr FINISHED!\n");
return 0;
}
When I run it, I get only the packet number count, i.e., I get no packet fields info, just the count of packets, with each number overriding the previous in the same place (no lines scroll happening).
Something like this, I guess for 4 packets:
tomas#ubuntu64:~$ sudo ./main
tshark: Lua: Error during loading:
[string "/usr/share/wireshark/init.lua"]:46: dofile has been disabled due to running Wireshark as superuser. See http://wiki.wireshark.org/CaptureSetup/CapturePrivileges for help in running Wireshark as an unprivileged user.
Running as user "root" and group "root". This could be dangerous.
Capturing on 'eth0'
4
But if I change in the C program the 'tshark' command argument by 'lsof', I get the standard output just fine.
tomas#ubuntu64:~$ sudo ./main
lsof: WARNING: can't stat() fuse.gvfsd-fuse file system /run/user/1000/gvfs
Output information may be incomplete.
SO: COMMAND PID TID USER FD TYPE DEVICE SIZE/OFF NODE NAME
SO: init 1 root cwd DIR 8,1 4096 2 /
SO: init 1 root rtd DIR 8,1 4096 2 /
SO: init 1 root txt REG 8,1 265848 791529 /sbin/init
SO: init 1 root mem REG 8,1 47712 824514 /lib/x86_64-linux-gnu/libnss_files-2.19.so
...
With this result, I'm assuming that there is something special with the way tshark sends the info to the standard output. Does anyone know about this behaviour?
I'm gonna check tshark source code, to see if it can clarify it.
Just a final note.
When I run tshark through the shell, I often get missing packet numbers like:
tomas#ubuntu64:~$ sudo tshark -i eth0 -R icmp -2 -T fields -e icmp.checksum -e icmp.seq
tshark: Lua: Error during loading:
[string "/usr/share/wireshark/init.lua"]:46: dofile has been disabled due to running Wireshark as superuser. See http://wiki.wireshark.org/CaptureSetup/CapturePrivileges for help in running Wireshark as an unprivileged user.
Running as user "root" and group "root". This could be dangerous.
Capturing on 'eth0'
0x0ee5 63045
1 0x8ae3 63046
2 0xcfdf 63047
3 0xe4d9 63048
4 0x9db7 63049
5 0x6798 63050
6 0x0175 63051
7 0x9e54 63052
0xa654 63052
9 0xe050 63053
0xe850 63053
11 0x8389 63054
0x8b89 63054
13 0x9b81 63055
0xa381 63055
Missing printed packet numbers 8, 10, 12, 14.
And when I redirect stdout to file, it does not send the count numbers:
tomas#ubuntu64:~$ sudo tshark -i eth0 -R icmp -2 -T fields -e icmp.checksum -e icmp.seq > kk
tomas#ubuntu64:~$ cat kk
0x2073 63321
0x2873 63321
0x7c6a 63322
Another clue that tshark is handling printed packet count and info differently.
Regards,
Tom
Well, even if I finally don't use this way of working with tshark, I think I found the option to use in order to popen tshark. From the manual page, option -l:
Flush the standard output after the information for each packet is printed. (This is not, strictly speaking, line-buffered if -V was
specified; however, it is the same as line-buffered if -V wasn't
specified, as only one line is printed for each packet, and, as -l is
normally used when piping a live capture to a program or script, so
that output for a packet shows up as soon as the packet is seen and
dissected, it should work just as well as true line-buffering. We do
this as a workaround for a deficiency in the Microsoft Visual C++ C
library.)
This may be useful when piping the output of TShark to another
program, as it means that the program to which the output is piped
will see the dissected data for a packet as soon as TShark sees the
packet and generates that output, rather than seeing it only when the
standard output buffer containing that data fills up.
I tested it, and it seems to work.
Just in case anyone needs it.
*/10 * * * * /usr/bin/flock -x -w 10 /tmp/craigslist.lock /usr/bin/lynx -width=120 -dump "http://sfbay.craigslist.org/search/roo/sfc?query=&srchType=A&minAsk=&maxAsk=1100&nh=6&nh=8&nh=16&nh=24&nh=17&nh=21&nh=22&nh=23&nh=27" | grep "sort by most recent" -A 53 > /home/winchell/apartments.txt
*/10 * * * * /usr/bin/flock -x -w 10 /tmp/craigslist.lock /usr/bin/php /home/winchell/apartments.php
This is a cron job. The second line php command seems to be executing even while lynx is writing to apartments.txt, and I don't see the reason. Is this correct usage assuming I'm trying to prevent read from apartments.txt while lynx/grep are writing to it? Thanks!
Your usage is not correct. Notice how your first cron job is a pipeline consisting of two commands:
/usr/bin/flock -x -w 10 /tmp/craigslist.lock /usr/bin/lynx -width=120 -dump
"http://sfbay.craigslist.org/search/roo/sfc?query=&srchType=A&minAsk=&maxAsk=1100&nh=6&nh=8&nh=16&nh=24&nh=17&nh=21&nh=22&nh=23&nh=27"
which is then piped to:
grep "sort by most recent" -A 53 > /home/winchell/apartments.txt
So the first command is locking a file but it's the second command that's writing to that file! The second command will happily execute without waiting for the lock.
One way to fix this would be to write the file while holding the lock:
lynx etc... | grep etc.. |
flock -x -w 10 /tmp/craigslist.lock tee /home/winchell/apartments.txt
The disadvantage of this approach is that lynx and grep run even if the file is locked. To prevent this, you will have to run the whole thing under the lock:
flock -x -w 10 /tmp/craigslock.lock sh -c "lynx etc... | grep etc... >thefile"
With this approach you will have to pay careful attention to quoting as the URL argument of lynx as it will require double quoting.
Finally: consider using curl or wget instead of lynx. lynx is meant for interactive usage!
I want to write a program that monitors syslog and performs an action when PPP authentication fails.
I think "tail -f /var/log/syslog" could help, but I'm not sure how to use it... probably using pipes?
I have found something similar written in bash, but I'm not sure how to implement it in C.
This is the bash method:
First create a named pipe using mkfifo:
$ mkfifo -p /home/mezgani/syslog.pipe
Make syslog.conf to points to this file:
*.info |/home/mezgani/syslog.pipe
Restart syslog:
$ sudo pkill -HUP syslogd
Create processing script that read the pipe
$ cat > foo
#!/bin/bash
cat /home/mezgani/syslog.pipe | while read input
do
# some stuff
echo ${input}
# ….
done
Finally I could found the solution!!
The solution was using named pipes!
First, I need to create a named pipe:
mkfifo /pipe
Then, I feed the pipe with the log info:
tail -f /var/log/syslog > /pipe
And then, I read the pipe from the C program using OPEN
int pipefd;
pipefd = open("/tmp/myFIFO", O_WRONLY);
Try to use inotify function. Using it you can monitor if a file or directory has changed.
I want to be able to log FFMPEG processes because I am trying to work out how long a minute of video takes to convert to help with capacity planning of my video encoding server. How do I enable logging and where is the log file saved. I have FFMPEG installed on a CentOS LAMP machine.
FFmpeg does not write to a specific log file, but rather sends its output to standard error. To capture that, you need to either
capture and parse it as it is generated
redirect standard error to a file and read that afterward the process is finished
Example for std error redirection:
ffmpeg -i myinput.avi {a-bunch-of-important-params} out.flv 2> /path/to/out.txt
Once the process is done, you can inspect out.txt.
It's a bit trickier to do the first option, but it is possible. (I've done it myself. So have others. Have a look around SO and the net for details.)
I found the below stuff in ffmpeg Docs. Hope this helps! :)
Reference: http://ffmpeg.org/ffmpeg.html#toc-Generic-options
‘-report’ Dump full command line and console output to a file named
program-YYYYMMDD-HHMMSS.log in the current directory. This file can be
useful for bug reports. It also implies -loglevel verbose.
Note: setting the environment variable FFREPORT to any value has the
same effect.
I find the answer.
1/First put in the presets, i have this example "Output format MPEG2 DVD HQ"
-vcodec mpeg2video -vstats_file MFRfile.txt -r 29.97 -s 352x480 -aspect 4:3 -b 4000k -mbd rd -trellis -mv0 -cmp 2 -subcmp 2 -acodec mp2 -ab 192k -ar 48000 -ac 2
If you want a report includes the commands -vstats_file MFRfile.txt into the presets like the example. this can make a report which it's ubicadet in the folder source of your file Source.
you can put any name if you want , i solved my problem "i write many times in this forum" reading a complete .docx about mpeg properties. finally i can do my progress bar reading this txt file generated.
Regards.
ffmpeg logs to stderr, and can log to a file with a different log-level from stderr. The -report command-line option doesn't give you control of the log file name or the log level, so setting the environment variable is preferable.
(-v is a synonym for -loglevel. Run ffmpeg -v help to see the levels. Run ffmpeg -h full | less to see EVERYTHING. Or consult the online docs, or their wiki pages like the h.264 encode guide).
#!/bin/bash
of=out.mkv
FFREPORT="level=32:file=$of.log" ffmpeg -v verbose -i src.mp4 -c:a copy -preset slower -c:v libx264 -crf 21 "$of"
That will trancode src.mp4 with x264, and set the log level for stderr to "verbose", and the log level for out.mkv.log to "status".
(AV_LOG_WARNING=24, AV_LOG_INFO=32, AV_LOG_VERBOSE=40, etc.). Support for this was added 2 years ago, so you need a non-ancient version of ffmpeg. (Always a good idea anyway, for security / bugfixes and speedups)
A few codecs, like -c:v libx265, write directly to stderr instead of using ffmpeg's logging infrastructure. So their log messages don't end up in the report file. I assume this is a bug / TODO-list item.
To log stderr, while still seeing it in a terminal, you can use tee(1).
If you use a log level that includes status line updates (the default -v info, or higher), they will be included in the log file, separated with ^M (carriage return aka \r). There's no log level that includes encoder stats (like SSIM) but not status-line updates, so the best option is probably to filter that stream.
If don't want to filter (e.g. so the fps / bitrate at each status-update interval is there in the file), you can use less -r to pass them through directly to your terminal so you can view the files cleanly. If you have .enc logs from several encodes that you want to flip through, less -r ++G *.enc works great. (++G means start at the end of the file, for all files). With single-key key bindings like . and , for next file and previous file, you can flip through some log files very nicely. (the default bindings are :n and :p).
If you do want to filter, sed 's/.*\r//' works perfectly for ffmpeg output. (In the general case, you need something like vt100.py, but not for just carriage returns). There are (at least) two ways to do this with tee + sed: tee to /dev/tty and pipe tee's output into sed, or use a process substitution to tee into a pipe to sed.
# pass stdout and stderr through to the terminal,
## and log a filtered version to a file (with only the last status-line update).
of="$1-x265.mkv"
ffmpeg -v info -i "$1" -c:a copy -c:v libx265 ... "$of" |& # pipe stdout and stderr
tee /dev/tty | sed 's/.*\r//' >> "$of.enc"
## or with process substitution where tee's arg will be something like /dev/fd/123
ffmpeg -v info -i "$1" -c:a copy -c:v libx265 ... "$of" |&
tee >(sed 's/.*\r//' >> "$of.enc")
For testing a few different encode parameters, you can make a function like this one that I used recently to test some stuff. I had it all on one line so I could easily up-arrow and edit it, but I'll un-obfuscate it here. (That's why there are ;s at the end of each line)
ffenc-testclip(){
# v should be set by the caller, to a vertical resolution. We scale to WxH, where W is a multiple of 8 (-vf scale=-8:$v)
db=0; # convenient to use shell vars to encode settings that you want to include in the filename and the ffmpeg cmdline
of=25s#21.15.${v}p.x265$pre.mkv;
[[ -e "$of.enc" ]]&&echo "$of.enc exists"&&return; # early-out if the file exists
# encode 25 seconds starting at 21m15s (or the keyframe before that)
nice -14 ffmpeg -ss $((21*60+15)) -i src.mp4 -t 25 -map 0 -metadata title= -color_primaries bt709 -color_trc bt709 -colorspace bt709 -sws_flags lanczos+print_info -c:a copy -c:v libx265 -b:v 1500k -vf scale=-8:$v -preset $pre -ssim 1 -x265-params ssim=1:cu-stats=1:deblock=$db:aq-mode=1:lookahead-slices=0 "$of" |&
tee /dev/tty | sed 's/.*\r//' >> "$of.enc";
}
# and use it with nested loops like this.
for pre in fast slow; do for v in 360 480 648 792;do ffenc-testclip ;done;done
less -r ++G *.enc # -r is useful if you didn't use sed
Note that it tests for existence of the output video file to avoid spewing extra garbage into the log file if it already exists. Even so, I used and append (>>) redirect.
It would be "cleaner" to write a shell function that took args instead of looking at shell variables, but this was convenient and easy to write for my own use. That's also why I saved space by not properly quoting all my variable expansions. ($v instead of "$v")
appears that if you add this to the command line:
-loglevel debug
or
-loglevel verbose
You get more verbose debugging output to the command line.
You can find more debugging info just simply adding the option -loglevel debug, full command will be
ffmpeg -i INPUT OUTPUT -loglevel debug -v verbose
You must declare the reportfile as variable for console.
Problem is all the Dokumentations you can find are not running so ..
I was give 1 day of my live to find the right way ....
Example: for batch/console
cmd.exe /K set FFREPORT=file='C:\ffmpeg\proto\test.log':level=32 && C:\ffmpeg\bin\ffmpeg.exe -loglevel warning -report -i inputfile f outputfile
Exemple Javascript:
var reortlogfile = "cmd.exe /K set FFREPORT=file='C:\ffmpeg\proto\" + filename + ".log':level=32 && C:\ffmpeg\bin\ffmpeg.exe" .......;
You can change the dir and filename how ever you want.
Frank from Berlin
If you just want to know how long it takes for the command to execute, you may consider using the time command. You for example use time ffmpeg -i myvideoofoneminute.aformat out.anotherformat