I'm a C programmer on linux.
I write a program that saves an image in /srv/ftp/preview.png which is updating frequently and i want to create a movie from this updates.
It's timestamp is important for me, e.g if image updates after 3.654 seconds i want movie show this update(frame) after 3.654 seconds too.
I searched in Internet for several hours but i can't find any solution.
I know about ffmpeg but it will convert images(and not one image) to movie without millisecond timestamp.
I found this Question but it seems is not useful in this case.
Is there any tool to do that? if not, please introduce an API in c to write a program myself
You can try to use inotify watch modification on the file and ffmpeg to append file to the movie:
#!/bin/bash
FRAMERATE=1
FILE="/path/to/image.jgp"
while true
do inotifywait -e modify "$FILE"
echo "file changed"
# create temp file name
TMP=$(mktemp)
# copy file
cp "$FILE" "$TMP$
# append copy file to movie
# from https://video.stackexchange.com/q/17228
# if movie already exist
if [ -f movie.mp4 ]
then
# append image to a new movie
ffmpeg -y -i movie.avi -loop 1 -f image2 -t $FRAMERATE -i "$TMP".jpg -f lavfi -t 3 -i anullsrc -filter_complex "[0:v] [1:v] concat=n=2:v=1 [v] " -map "[v]" newmovie.avi
# replace old by new movie
mv newmovie.mp4 movie.mp4
else
#create a movie from one image
ffmpeg -framerate 1 -t $FRAMERATE -i "$TMP" movie.mp4
fi
rm "$TMP"
done
This script must certainly be adapted, (in particular if your framerate is high) but I think you can try to play with it.
One bad thing also that the movie creation will become slower and slower because the movie becomes bigger.
You should to store images of a certain time duration in a directory and convert all at once (like once an hour/day)
If you want to serve a stream instead of creating a video file, you can look at https://stackoverflow.com/a/31705978/1212012
Related
youtube-dl can take some time parsing remote sites when called multiple times.
EDIT0 : I want to fetch multiple properties (here fileNames and remoteFileSizes) output by youtube-dl without having to run it multiple times.
I use those 2 properties to compare the local file size and ${remoteFileSizes[$i]} to tell if the file is finished downloading.
$ youtube-dl --restrict-filenames -o "%(title)s__%(format_id)s__%(id)s.%(ext)s" -f m4a,18,webm,251 -s -j https://www.youtube.com/watch?v=UnZbjvyzteo 2>errors_youtube-dl.log | jq -r ._filename,.filesize | paste - - > input_data.txt
$ cat input_data.txt
Alan_Jackson_-_I_Want_To_Stroll_Over_Heaven_With_You_Live__18__UnZbjvyzteo__youtube_com.mp4 8419513
Alan_Jackson_-_I_Want_To_Stroll_Over_Heaven_With_You_Live__250__UnZbjvyzteo__youtube_com.webm 1528955
Alan_Jackson_-_I_Want_To_Stroll_Over_Heaven_With_You_Live__140__UnZbjvyzteo__youtube_com.m4a 2797366
Alan_Jackson_-_I_Want_To_Stroll_Over_Heaven_With_You_Live__244__UnZbjvyzteo__youtube_com.webm 8171725
I want the first column in the fileNames array and the second column in the remoteFileSizes.
For the time being, I use a while read loop, but when this loop is finished my two arrays are lost :
$ fileNames=()
$ remoteFileSizes=()
$ cat input_data.txt | while read fileName remoteFileSize; do \
fileNames+=($fileName); \
remoteFileSizes+=($remoteFileSize); \
done
$ for fileNames in "${fileNames[#]}"; do \
echo PROCESSING....; \
done
$ echo "=> fileNames[0] = ${fileNames[0]}"
=> fileNames[0] =
$ echo "=> remoteFileSizes[0] = ${remoteFileSizes[0]}"
=> remoteFileSizes[0] =
$
Is it possible to assign two bash arrays with a single command ?
You assign variables in a subshell, so they are not visible in the parent shell. Read https://mywiki.wooledge.org/BashFAQ/024 . Remove the cat and do a redirection to solve your problem.
while IFS=$'\t' read -r fileName remoteFileSize; do
fileNames+=("$fileName")
remoteFileSizes+=("$remoteFileSize")
done < input_data.txt
You might also interest yourself in https://mywiki.wooledge.org/BashFAQ/001.
For what it's worth, if you're looking for specific/bespoke functionality from youtube-dl, I recommend creating your own python scripts using the 'embedded' approach: https://github.com/ytdl-org/youtube-dl/blob/master/README.md#embedding-youtube-dl
You can set your own signal for when a download is finished (text/chime/mail/whatever) and track downloads without having to compare file sizes.
I try to write a wrapper script for songbook generation using lilypond, latex and sejda-console (for the pdf part). Everything works so far, but I have a problem with sejda that is giving me nuts.
Here is the relevant part of my code:
for %%i in (%f%) do (
sejda-console.bat extractbybookmarks -f ".\%%~ni.pdf" -o "export\%%~ni\%title%.pdf" -l 2 -p [BOOKMARK_NAME] -e "%title%" --overwrite
)
where f is a ";"-separated list of files. This command works for the first file, but fails for all others. I can't find any difference between the commands that sejda receives. Here is my console output:
make_sheet.bat -t "Live it up" --supress *.lytex
Configuring Sejda 3.2.30
Starting execution with arguments: 'extractbybookmarks -f .\book_drums.pdf -o export\book_drums\Live it up.pdf -l 2 -p [BOOKMARK_NAME] -e Live it up --overwrite'
Java version: '1.8.0_151'
Validating parameters.
Starting task (org.sejda.impl.sambox.ExtractByOutlineTask#28701274) execution.
Opening C:\Users\skr1_\Desktop\Tools\Songbook\Sample\out\.\book_drums.pdf
Retrieving outline information for level 2 and match regex Live it up
Starting extraction by outline, level 2 and match regex Live it up
Found 0 inherited images and 0 inherited fonts potentially unused
Starting extracting Live it up pages 9 9
Created output temporary buffer C:\Users\skr1_\Desktop\Tools\Songbook\Sample\out\export\book_drums\.sejdaTmp2789047920522272436.tmp
Appended relevant outline items
Filtering annotations
Skipped acroform merge, nothing to merge
Ending extracting Live it up
Task progress: 0% done
Moving C:\Users\skr1_\Desktop\Tools\Songbook\Sample\out\export\book_drums\.sejdaTmp2789047920522272436.tmp to C:\Users\skr1_\Desktop\Tools\Songbook\Sample\out\export\book_drums\Live it up.pdf.
Extraction completed and outputs written to org.sejda.model.output.FileOrDirectoryTaskOutput#478190fc[C:\Users\skr1_\Desktop\Tools\Songbook\Sample\out\export\book_drums\Live it up.pdf]
Task (org.sejda.impl.sambox.ExtractByOutlineTask#28701274) executed in 0 seconds
Completed execution
C:\Users\skr1_\Desktop\Tools\Songbook\Sample>(sejda-console.bat extractbybookmarks -f ".\book_general.pdf" -o "export\book_general\Live it up.pdf" -l 2 -p [BOOKMARK_NAME] -e "Live it up" --overwrite )
Configuring Sejda 3.2.30
Starting execution with arguments: 'extractbybookmarks -f .\book_general.pdf -o export\book_general\Live it up.pdf -l 2 -p [BOOKMARK_NAME] -e Live it up --overwrite'
Java version: '1.8.0_151'
Invalid value (File '.\book_general.pdf' does not exist): --files -f value... : pdf files to operate on. A list of existing pdf files (EX. -f /tmp/file1.pdf or -f /tmp/password_protected_file2.pdf:secret123) (required)
Invalid value (File '.\book_general.pdf' does not exist): --files -f value... : pdf files to operate on. A list of existing pdf files (EX. -f /tmp/file1.pdf or -f /tmp/password_protected_file2.pdf:secret123) (required)
C:\Users\skr1_\Desktop\Tools\Songbook\Sample>(sejda-console.bat extractbybookmarks -f ".\book_guitar.pdf" -o "export\book_guitar\Live it up.pdf" -l 2 -p [BOOKMARK_NAME] -e "Live it up" --overwrite )
Configuring Sejda 3.2.30
Starting execution with arguments: 'extractbybookmarks -f .\book_guitar.pdf -o export\book_guitar\Live it up.pdf -l 2 -p [BOOKMARK_NAME] -e Live it up --overwrite'
Java version: '1.8.0_151'
Invalid value (File '.\book_guitar.pdf' does not exist): --files -f value... : pdf files to operate on. A list of existing pdf files (EX. -f /tmp/file1.pdf or -f /tmp/password_protected_file2.pdf:secret123) (required)
Invalid value (File '.\book_guitar.pdf' does not exist): --files -f value... : pdf files to operate on. A list of existing pdf files (EX. -f /tmp/file1.pdf or -f /tmp/password_protected_file2.pdf:secret123) (required)
Even worse, if I copy the commands that sejda receives and paste them as arguments for a new command, everything works fine.
I suspect that something is happening with the working directory in between, but I don't get it.
Also, note that the output includes the command for subsequent passes of the for-loop (starting with "(sejda-console.bat ...") though echo is off. It is not included for the first run, however.
I'm not an expert with programming, especially not with batch, and any help would be very appreciated.
I'd suggest that sejda.bat is changing the current directory.
Try
pushd
call sejda.bat ...
popd
I want to be able to log FFMPEG processes because I am trying to work out how long a minute of video takes to convert to help with capacity planning of my video encoding server. How do I enable logging and where is the log file saved. I have FFMPEG installed on a CentOS LAMP machine.
FFmpeg does not write to a specific log file, but rather sends its output to standard error. To capture that, you need to either
capture and parse it as it is generated
redirect standard error to a file and read that afterward the process is finished
Example for std error redirection:
ffmpeg -i myinput.avi {a-bunch-of-important-params} out.flv 2> /path/to/out.txt
Once the process is done, you can inspect out.txt.
It's a bit trickier to do the first option, but it is possible. (I've done it myself. So have others. Have a look around SO and the net for details.)
I found the below stuff in ffmpeg Docs. Hope this helps! :)
Reference: http://ffmpeg.org/ffmpeg.html#toc-Generic-options
‘-report’ Dump full command line and console output to a file named
program-YYYYMMDD-HHMMSS.log in the current directory. This file can be
useful for bug reports. It also implies -loglevel verbose.
Note: setting the environment variable FFREPORT to any value has the
same effect.
I find the answer.
1/First put in the presets, i have this example "Output format MPEG2 DVD HQ"
-vcodec mpeg2video -vstats_file MFRfile.txt -r 29.97 -s 352x480 -aspect 4:3 -b 4000k -mbd rd -trellis -mv0 -cmp 2 -subcmp 2 -acodec mp2 -ab 192k -ar 48000 -ac 2
If you want a report includes the commands -vstats_file MFRfile.txt into the presets like the example. this can make a report which it's ubicadet in the folder source of your file Source.
you can put any name if you want , i solved my problem "i write many times in this forum" reading a complete .docx about mpeg properties. finally i can do my progress bar reading this txt file generated.
Regards.
ffmpeg logs to stderr, and can log to a file with a different log-level from stderr. The -report command-line option doesn't give you control of the log file name or the log level, so setting the environment variable is preferable.
(-v is a synonym for -loglevel. Run ffmpeg -v help to see the levels. Run ffmpeg -h full | less to see EVERYTHING. Or consult the online docs, or their wiki pages like the h.264 encode guide).
#!/bin/bash
of=out.mkv
FFREPORT="level=32:file=$of.log" ffmpeg -v verbose -i src.mp4 -c:a copy -preset slower -c:v libx264 -crf 21 "$of"
That will trancode src.mp4 with x264, and set the log level for stderr to "verbose", and the log level for out.mkv.log to "status".
(AV_LOG_WARNING=24, AV_LOG_INFO=32, AV_LOG_VERBOSE=40, etc.). Support for this was added 2 years ago, so you need a non-ancient version of ffmpeg. (Always a good idea anyway, for security / bugfixes and speedups)
A few codecs, like -c:v libx265, write directly to stderr instead of using ffmpeg's logging infrastructure. So their log messages don't end up in the report file. I assume this is a bug / TODO-list item.
To log stderr, while still seeing it in a terminal, you can use tee(1).
If you use a log level that includes status line updates (the default -v info, or higher), they will be included in the log file, separated with ^M (carriage return aka \r). There's no log level that includes encoder stats (like SSIM) but not status-line updates, so the best option is probably to filter that stream.
If don't want to filter (e.g. so the fps / bitrate at each status-update interval is there in the file), you can use less -r to pass them through directly to your terminal so you can view the files cleanly. If you have .enc logs from several encodes that you want to flip through, less -r ++G *.enc works great. (++G means start at the end of the file, for all files). With single-key key bindings like . and , for next file and previous file, you can flip through some log files very nicely. (the default bindings are :n and :p).
If you do want to filter, sed 's/.*\r//' works perfectly for ffmpeg output. (In the general case, you need something like vt100.py, but not for just carriage returns). There are (at least) two ways to do this with tee + sed: tee to /dev/tty and pipe tee's output into sed, or use a process substitution to tee into a pipe to sed.
# pass stdout and stderr through to the terminal,
## and log a filtered version to a file (with only the last status-line update).
of="$1-x265.mkv"
ffmpeg -v info -i "$1" -c:a copy -c:v libx265 ... "$of" |& # pipe stdout and stderr
tee /dev/tty | sed 's/.*\r//' >> "$of.enc"
## or with process substitution where tee's arg will be something like /dev/fd/123
ffmpeg -v info -i "$1" -c:a copy -c:v libx265 ... "$of" |&
tee >(sed 's/.*\r//' >> "$of.enc")
For testing a few different encode parameters, you can make a function like this one that I used recently to test some stuff. I had it all on one line so I could easily up-arrow and edit it, but I'll un-obfuscate it here. (That's why there are ;s at the end of each line)
ffenc-testclip(){
# v should be set by the caller, to a vertical resolution. We scale to WxH, where W is a multiple of 8 (-vf scale=-8:$v)
db=0; # convenient to use shell vars to encode settings that you want to include in the filename and the ffmpeg cmdline
of=25s#21.15.${v}p.x265$pre.mkv;
[[ -e "$of.enc" ]]&&echo "$of.enc exists"&&return; # early-out if the file exists
# encode 25 seconds starting at 21m15s (or the keyframe before that)
nice -14 ffmpeg -ss $((21*60+15)) -i src.mp4 -t 25 -map 0 -metadata title= -color_primaries bt709 -color_trc bt709 -colorspace bt709 -sws_flags lanczos+print_info -c:a copy -c:v libx265 -b:v 1500k -vf scale=-8:$v -preset $pre -ssim 1 -x265-params ssim=1:cu-stats=1:deblock=$db:aq-mode=1:lookahead-slices=0 "$of" |&
tee /dev/tty | sed 's/.*\r//' >> "$of.enc";
}
# and use it with nested loops like this.
for pre in fast slow; do for v in 360 480 648 792;do ffenc-testclip ;done;done
less -r ++G *.enc # -r is useful if you didn't use sed
Note that it tests for existence of the output video file to avoid spewing extra garbage into the log file if it already exists. Even so, I used and append (>>) redirect.
It would be "cleaner" to write a shell function that took args instead of looking at shell variables, but this was convenient and easy to write for my own use. That's also why I saved space by not properly quoting all my variable expansions. ($v instead of "$v")
appears that if you add this to the command line:
-loglevel debug
or
-loglevel verbose
You get more verbose debugging output to the command line.
You can find more debugging info just simply adding the option -loglevel debug, full command will be
ffmpeg -i INPUT OUTPUT -loglevel debug -v verbose
You must declare the reportfile as variable for console.
Problem is all the Dokumentations you can find are not running so ..
I was give 1 day of my live to find the right way ....
Example: for batch/console
cmd.exe /K set FFREPORT=file='C:\ffmpeg\proto\test.log':level=32 && C:\ffmpeg\bin\ffmpeg.exe -loglevel warning -report -i inputfile f outputfile
Exemple Javascript:
var reortlogfile = "cmd.exe /K set FFREPORT=file='C:\ffmpeg\proto\" + filename + ".log':level=32 && C:\ffmpeg\bin\ffmpeg.exe" .......;
You can change the dir and filename how ever you want.
Frank from Berlin
If you just want to know how long it takes for the command to execute, you may consider using the time command. You for example use time ffmpeg -i myvideoofoneminute.aformat out.anotherformat
I execute the following command line
ffmpeg.exe
-i C:\Beema\video-source\DO_U_BEEMA176x144short.avi
-i C:\Beema\video-source\DO_U_BEEMA176x144short.avi
-i C:\Beema\temp\9016730-51056331-stitcheds.avi
-i C:\Beema\video-source\GOTTA_BEEMA176x144short.avi
-y -ac 1 -r 24 -b 25K
C:\Beema\video-out\9a062fb6-d448-48fe-b006-a85d51adf8a1.mpg
The output file in video-out ends up having a single copy of DO_U_BEEMA. I do not understand why ffmpeg is not concatenating.
Any help is dramatically appreciated,
mencoder -oac copy -ovc copy file1 file2 file3 … -o final_movie.mpg
Have you tried with mencoder?
Also are all of the files the same bitrate and dimensions? If not your going to need to make sure all of the video files are identical in these two areas before attempting to combine. It also appears your attempting to combine .avi's with a single .mpg, you'll most likely want to convert the .mpg to a similar format when re-encoding.
Hope this helps.
If it has C:\, it's Windows. use:
copy video1 + video2 + video3
and add more + videoN instances until you get there.
Here is the command that will work for you first cat the files then pipe to ffmpeg
cat C:\Beema\video-source\DO_U_BEEMA176x144short.avi C:\Beema\video-source\DO_U_BEEMA176x144short.avi C:\Beema\temp\9016730-51056331-stitcheds.avi
C:\Beema\video-source\GOTTA_BEEMA176x144short.avi | ffmpeg -f mpeg -i - C:\Beema\video-out\9a062fb6-d448-48fe-b006-a85d51adf8a1.mpg
-i - is important "-" this is the piped input to ffmpeg
Cheers.
I guess ffmpeg cann't do that. It usually takes only one input file. Trying to cat files to input may result in a lump togather, but i guess it won't stitch properly.
Best bet is mencoder or using transcode
I want to programmatically create a SHA1 checksum of audio files (MP3, Ogg Vorbis, Flac).
The requirement is that the checksum should be stable even if the header (eg. ID3) changes.
Note: The audio files don't have CRCs
This is what I tried by now:
1) Reading + Hashing all MPEG frames using Perl and MPEG::Audio::Frame
my $sha1 = Digest::SHA1->new;
while (my $frame = MPEG::Audio::Frame->read(\*FH)) {
$sha1->add($frame->content());
}
2) Decoding + Hashing all MPEG frames using Python and libmad (pymad)
mf = mad.MadFile(path)
sha1 = hashlib.sha1()
while 1:
buf = mf.read()
if (buf is None):
break
sha1.update(buf)
3) Using mp3cat
> mp3cat - - < file.mp3 | sha1sum
However, none of those methods provided a stable checksum. Namely, in some cases the checksum changed after retagging the file with picard.
Are there any libraries that already provide what I want?
I don't care about the programming language…
Update:
I debugged the case a bit further.
The libmad checksum inconsitency seems to happen in cases where libmad gets some decoding errors, like "Huffman data overrun (0x0238)".
As this really happens on many of the mp3 files I'm not sure if it really indicates a broken file…
If you are looking for stable hashes for the actual music you might want to look at libOFA. Your current methods will give you different results because the formats can have embedded tags. Also if you want two different files with the same song to return the same hash you need to regard things like bitrate and sample frequencies.
libOFA on the other hand can give you a stable hash that can be used between formats and different encodings. Might be what you want?
I needed tools to quickly check if my MP3/OGG library is still valid.
For MP3 I found mp3md5.py (http://snipplr.com/view/4025/mp3-checksum-in-id3-tag/) which does the job, but no simple tool for OGG Vorbis, but I coded a little bash script to do this for me.
Both tools should tolerate modifications of the comment/ID3Tag.
#!/bin/bash
# This bash script appends an MD5SUM to the vorbiscomment and/or verifies it if it exists
# Later modification of the vorbis comment does not alter the MD5SUM
# Julian M.K.
FILE="$1"
if [[ ! -f "$FILE" || ! -r "$FILE" || ! -w "$FILE" ]] ; then
echo "File $FILE" does not exist or is not readable or writable
exit 1
fi
OLDCRC=`vorbiscomment "$FILE" | grep ^CRC=|cut -d "=" -f 2`
NEWCRC=`ogginfo "$FILE" |grep "Total data length:" |cut -d ":" -f 2 | md5sum |cut -d " " -f 1`
if [[ "$OLDCRC" == "" ]] ; then
echo "ADDED $FILE $NEWCRC"
vorbiscomment -a -t "CRC=$NEWCRC" "$FILE"
# rewrite CRC to get proper data length, I dont know why this is necessary
NEWCRC=`ogginfo "$FILE" |grep "Total data length:" |cut -d ":" -f 2 | md5sum |cut -d " " -f 1`
vorbiscomment -w -t "CRC=$NEWCRC" "$FILE"
elif [[ "$OLDCRC" == "$NEWCRC" ]] ; then
echo "VERIFIED $FILE"
else
echo "FAILURE $FILE -- $OLDCRC - $NEWCRC"
fi
There is an easy stable way to do it. Just make a copy of the file and remove all the tags from it (e.g. using mutagen.id3) and take the hashsum of the resulting file.
The only disadvantage of this method is its performance.
Bene, If I were you, (And I am in the process of working on something very similar to what you want to do), I would hash the mp3 data block. (Extract it to raw data first, and write it out to disk, so you know what you are dealing with). Then, modify the ID3 tag. Hash your data again. Now, if it changes, compare your two sets of raw data and find out WHERE it changed. Chances are, you might be over-stepping a boundary somewhere. If I recall, MP3 files start with something like FF F8. Well, at least the frame does.
I'm interested in your findings, as I'm still writing all my code to deal with the finger prints, etc, and haven't gotten to the actual hashing yet.
Update many years later:
See my answer here to a very similar question. It turns out that ffmpeg actually supports doing checksums of the individual streams. To get the md5 hash of only the audio stream:
ffmpeg -i "$filename" -map 0:a -codec copy -f md5 "$filename.md5"
There is also support for other hash formats with the generic -f hash format, or for doing it per frame with -f framemd5.
I'm trying to do the same thing. I used MD5 instead of SHA1. I started to export audio checksums using mp3tag (www.mp3tag.de/en/); then made a Perl script similar to yours to do the same thing. Then I removed all tags from my test file, and the audio checksum remained the same.
This is the script:
use MPEG::Audio::Frame;
use Digest::MD5 qw(md5_hex);
use strict;
my $file = 'E:\Music\MP3\Russensoul\01 - 5nizza , Soldat (Russensoul - Russensoul).mp3';
my $mp3tag_audio_md5 = lc '2EDFBD62995A46A45CEEC08C1F303486';
my $md5 = Digest::MD5->new;
open(FILE, $file) or die "Cannot open $file : $!\n";
binmode FILE;
while(my $frame = MPEG::Audio::Frame->read(\*FILE)){
$md5->add($frame->asbin);
}
print '$md5->hexdigest : ', $md5->hexdigest, "\n",
'mp3tag_audio_md5 : ', $mp3tag_audio_md5, "\n",
;
Is it possible that whatever you use to modify your tags sometimes also modifies mp3 headers?