I am writing a script in Lua 5.1 for use with a game engine (EDGE).
I need my script to copy about 20 files into a .miz file (which is really a zipped folder with a set structure) and navigate that structure and copy those files in from a non-zipped folder on the hard drive.
Because Windows 11 it the future I need to use NanaZip rather than 7z which isn't W11 supported.
However, all the examples I've found are for using LUA to zip up files, not insert non-zipped files INTO a zip file without unzipping it.
Is this even possible?
Similar to #koyaanisqatsi I tried it with 7z. You didn't comment on our question on why 7z should be avoided nor whether you are even allowed to use os.execute, but it should provide a good starting point:
os.execute("7z a yourZip.zip yourFile.png")
Where a is the flag for Add.
See the manual for other flags like compression: https://linux.die.net/man/1/7z
Windows 11 also have tar that have the option r and u
D:\temp>tar h
tar(bsdtar): manipulate archive files
First option must be a mode specifier:
-c Create -r Add/Replace -t List -u Update -x Extract
Common Options:
-b # Use # 512-byte records per I/O block
-f <filename> Location of archive (default \\.\tape0)
-v Verbose
-w Interactive
Create: tar -c [options] [<file> | <dir> | #<archive> | -C <dir> ]
<file>, <dir> add these items to archive
-z, -j, -J, --lzma Compress archive with gzip/bzip2/xz/lzma
--format {ustar|pax|cpio|shar} Select archive format
--exclude <pattern> Skip files that match pattern
-C <dir> Change to <dir> before processing remaining files
#<archive> Add entries from <archive> to output
List: tar -t [options] [<patterns>]
<patterns> If specified, list only entries that match
Extract: tar -x [options] [<patterns>]
<patterns> If specified, extract only entries that match
-k Keep (don't overwrite) existing files
-m Don't restore modification times
-O Write entries to stdout, don't restore to disk
-p Restore permissions (including ACLs, owner, file flags)
bsdtar 3.5.2 - libarchive 3.5.2 zlib/1.2.5.f-ipp bz2lib/1.0.6
( Above cmd.exe was opened from Lua with: os.execute('cmd') )
You can extract a ZIP with it but not creating one as far as i know.
(tar -xf archive.zip)
But is it a Problem for you to use TAR instead of ZIP?
I am running a program that a given points do a system call to zip some files.
The first time it runs properly, but the next time (and also the next ones) it fails with a -1 error code.
sprintf(zip, "zip -q -T -m -r %s/datsiter%d.zip %s/*%d.dat", dir, it_number, dir, it_number);
ierr = system(zip);
printf("Output:%s %d\n ",zip, ierr);
The output is:
Output:zip -q -T -m -r results/datsiter0.zip results/*0.dat 0
Output:zip -q -T -m -r results/datsiter1.zip results/*1.dat -1
Output:zip -q -T -m -r results/datsiter2.zip results/*2.dat -1
If after the run I execute the commands manually, it works as expected.
EDIT
Following kaylum suggestion I added perror("Error: "); just after system call and got the key. A nice Cannot allocate memory that I'll have to check carefully since I'm working in a cluster with a queue system.
Following kaylum suggestion I added perror("Error: "); just after system call and got the key. A nice Cannot allocate memory that I'll have to check carefully since I'm working in a cluster with a queue system.
I am working on embedded linux. I am trying to protect my rootfs by making it read only and mount fs file from the sdcard over the root.
I need both fs to be merged.
Any writes can be redirected to the fs file onto the sdcard.
Reading from the read only rootfs still possible.
I tried the following:
$ cd /media/sdcard
$ mount userfs /
$ cd /
$ echo a > a.txt
But I receive error:
-sh: a.txt: Read-only file system
Can any one help me to implement the needed functionality ?
To complete Ross answer, this is how I added overlayfs for /var/log:
add_overlayfs_mount() {
mkdir -p ${IMAGE_ROOTFS}/data/overlay/log
mkdir -p ${IMAGE_ROOTFS}/data/work/log
echo '/dev/sda4 /data ext4 defaults 0 0' >> ${IMAGE_ROOTFS}/etc/fstab
echo 'ofslog /var/log overlay defaults,x-systemd.requires=data,lowerdir=/var/log,upperdir=/data/overlay/log,workdir=/data/work/log 0 2' >> ${IMAGE_ROOTFS}/etc/fstab
}
ROOTFS_POSTPROCESS_COMMAND += "add_overlayfs_mount ; "
You can also use VOLATILE_BINDS in some situations:
VOLATILE_BINDS_append = " \
/data/etc/hostname /etc/hostname \n\
"
Yes, overlayfs is exactly what you want.
I accidentally removed a wrong folder using rm -rf, every tool I've tried tell me either I have no hard disk or that the filesystem is not found.
When I type df I get :
Filesystem Size Used Avail Use% Mounted on
/dev/simfs 25G 7.6G 18G 31% /
none 256M 12K 256M 1% /dev
But if I type debugfs -w /dev/simfs I get the error :
/dev/simfs: No such file or directory while opening filesystem
I'm using a VPS hosted by OVH, what's wrong with that filesystem ?
I have the same scenario/issue. To take the conversation a step further, I've also tried
tune2fs -l /dev/simfs
which outputs
tune2fs 1.42.9 (4-Feb-2014)
tune2fs: No such file or directory while trying to open /dev/simfs
Couldn't find valid filesystem superblock.
You'll get the same invalid superblock error if you try to scan an entire disk eg /dev/sda
I want to be able to log FFMPEG processes because I am trying to work out how long a minute of video takes to convert to help with capacity planning of my video encoding server. How do I enable logging and where is the log file saved. I have FFMPEG installed on a CentOS LAMP machine.
FFmpeg does not write to a specific log file, but rather sends its output to standard error. To capture that, you need to either
capture and parse it as it is generated
redirect standard error to a file and read that afterward the process is finished
Example for std error redirection:
ffmpeg -i myinput.avi {a-bunch-of-important-params} out.flv 2> /path/to/out.txt
Once the process is done, you can inspect out.txt.
It's a bit trickier to do the first option, but it is possible. (I've done it myself. So have others. Have a look around SO and the net for details.)
I found the below stuff in ffmpeg Docs. Hope this helps! :)
Reference: http://ffmpeg.org/ffmpeg.html#toc-Generic-options
‘-report’ Dump full command line and console output to a file named
program-YYYYMMDD-HHMMSS.log in the current directory. This file can be
useful for bug reports. It also implies -loglevel verbose.
Note: setting the environment variable FFREPORT to any value has the
same effect.
I find the answer.
1/First put in the presets, i have this example "Output format MPEG2 DVD HQ"
-vcodec mpeg2video -vstats_file MFRfile.txt -r 29.97 -s 352x480 -aspect 4:3 -b 4000k -mbd rd -trellis -mv0 -cmp 2 -subcmp 2 -acodec mp2 -ab 192k -ar 48000 -ac 2
If you want a report includes the commands -vstats_file MFRfile.txt into the presets like the example. this can make a report which it's ubicadet in the folder source of your file Source.
you can put any name if you want , i solved my problem "i write many times in this forum" reading a complete .docx about mpeg properties. finally i can do my progress bar reading this txt file generated.
Regards.
ffmpeg logs to stderr, and can log to a file with a different log-level from stderr. The -report command-line option doesn't give you control of the log file name or the log level, so setting the environment variable is preferable.
(-v is a synonym for -loglevel. Run ffmpeg -v help to see the levels. Run ffmpeg -h full | less to see EVERYTHING. Or consult the online docs, or their wiki pages like the h.264 encode guide).
#!/bin/bash
of=out.mkv
FFREPORT="level=32:file=$of.log" ffmpeg -v verbose -i src.mp4 -c:a copy -preset slower -c:v libx264 -crf 21 "$of"
That will trancode src.mp4 with x264, and set the log level for stderr to "verbose", and the log level for out.mkv.log to "status".
(AV_LOG_WARNING=24, AV_LOG_INFO=32, AV_LOG_VERBOSE=40, etc.). Support for this was added 2 years ago, so you need a non-ancient version of ffmpeg. (Always a good idea anyway, for security / bugfixes and speedups)
A few codecs, like -c:v libx265, write directly to stderr instead of using ffmpeg's logging infrastructure. So their log messages don't end up in the report file. I assume this is a bug / TODO-list item.
To log stderr, while still seeing it in a terminal, you can use tee(1).
If you use a log level that includes status line updates (the default -v info, or higher), they will be included in the log file, separated with ^M (carriage return aka \r). There's no log level that includes encoder stats (like SSIM) but not status-line updates, so the best option is probably to filter that stream.
If don't want to filter (e.g. so the fps / bitrate at each status-update interval is there in the file), you can use less -r to pass them through directly to your terminal so you can view the files cleanly. If you have .enc logs from several encodes that you want to flip through, less -r ++G *.enc works great. (++G means start at the end of the file, for all files). With single-key key bindings like . and , for next file and previous file, you can flip through some log files very nicely. (the default bindings are :n and :p).
If you do want to filter, sed 's/.*\r//' works perfectly for ffmpeg output. (In the general case, you need something like vt100.py, but not for just carriage returns). There are (at least) two ways to do this with tee + sed: tee to /dev/tty and pipe tee's output into sed, or use a process substitution to tee into a pipe to sed.
# pass stdout and stderr through to the terminal,
## and log a filtered version to a file (with only the last status-line update).
of="$1-x265.mkv"
ffmpeg -v info -i "$1" -c:a copy -c:v libx265 ... "$of" |& # pipe stdout and stderr
tee /dev/tty | sed 's/.*\r//' >> "$of.enc"
## or with process substitution where tee's arg will be something like /dev/fd/123
ffmpeg -v info -i "$1" -c:a copy -c:v libx265 ... "$of" |&
tee >(sed 's/.*\r//' >> "$of.enc")
For testing a few different encode parameters, you can make a function like this one that I used recently to test some stuff. I had it all on one line so I could easily up-arrow and edit it, but I'll un-obfuscate it here. (That's why there are ;s at the end of each line)
ffenc-testclip(){
# v should be set by the caller, to a vertical resolution. We scale to WxH, where W is a multiple of 8 (-vf scale=-8:$v)
db=0; # convenient to use shell vars to encode settings that you want to include in the filename and the ffmpeg cmdline
of=25s#21.15.${v}p.x265$pre.mkv;
[[ -e "$of.enc" ]]&&echo "$of.enc exists"&&return; # early-out if the file exists
# encode 25 seconds starting at 21m15s (or the keyframe before that)
nice -14 ffmpeg -ss $((21*60+15)) -i src.mp4 -t 25 -map 0 -metadata title= -color_primaries bt709 -color_trc bt709 -colorspace bt709 -sws_flags lanczos+print_info -c:a copy -c:v libx265 -b:v 1500k -vf scale=-8:$v -preset $pre -ssim 1 -x265-params ssim=1:cu-stats=1:deblock=$db:aq-mode=1:lookahead-slices=0 "$of" |&
tee /dev/tty | sed 's/.*\r//' >> "$of.enc";
}
# and use it with nested loops like this.
for pre in fast slow; do for v in 360 480 648 792;do ffenc-testclip ;done;done
less -r ++G *.enc # -r is useful if you didn't use sed
Note that it tests for existence of the output video file to avoid spewing extra garbage into the log file if it already exists. Even so, I used and append (>>) redirect.
It would be "cleaner" to write a shell function that took args instead of looking at shell variables, but this was convenient and easy to write for my own use. That's also why I saved space by not properly quoting all my variable expansions. ($v instead of "$v")
appears that if you add this to the command line:
-loglevel debug
or
-loglevel verbose
You get more verbose debugging output to the command line.
You can find more debugging info just simply adding the option -loglevel debug, full command will be
ffmpeg -i INPUT OUTPUT -loglevel debug -v verbose
You must declare the reportfile as variable for console.
Problem is all the Dokumentations you can find are not running so ..
I was give 1 day of my live to find the right way ....
Example: for batch/console
cmd.exe /K set FFREPORT=file='C:\ffmpeg\proto\test.log':level=32 && C:\ffmpeg\bin\ffmpeg.exe -loglevel warning -report -i inputfile f outputfile
Exemple Javascript:
var reortlogfile = "cmd.exe /K set FFREPORT=file='C:\ffmpeg\proto\" + filename + ".log':level=32 && C:\ffmpeg\bin\ffmpeg.exe" .......;
You can change the dir and filename how ever you want.
Frank from Berlin
If you just want to know how long it takes for the command to execute, you may consider using the time command. You for example use time ffmpeg -i myvideoofoneminute.aformat out.anotherformat