I have a list of files that I want to zip but I also have a list to exclude files and do not want them to be included in the zip archive.
so I have created a exclude.lst file and it has absolute path and filenames in it.
sample exclude file
/home/logs/apache/access.log
/home/logs/tomcat/catalina.out
but after using the below command, the zip command is not excluding the files rather archiving them.
zip archives.2012.zip /home/logs/ -x#exclude.lst
how can I overcome this ? and is there any other way to archive files by excluding the above files.
Instead of creating exclude.lst file, I'm assigning all the exclude files to a variable and passing those to the -x option in the zip.
For example
do_not_archive=/home/logs/apache/access.log /home/logs/tomcat/catalina.out
Then use zip as shown below
zip archives.2012.zip /home/logs/ -x $do_not_archive
Old question, so zip may have changed since then, but from the man pages:
$man zip | grep -A2 exclude
Also possible:
zip -r foo foo -x#exclude.lst
which will include the contents of foo in foo.zip while excluding all the files that match the patterns in the file exclude.lst.
I've confirmed that this works in Ubuntu 18, with each pattern/filename on a separate line (haven't tried separated by spaces)
Just a short addition to CBR's answer for the case you have a bunch of files to exclude (in my case files bigger than 10MB):
do_not_archive=$(find relative/path/to/directory -type f -size +10000000c)
zip -r backup_without_files_bigger_than_10mb.zip relative/path/to/directory -x $do_not_archive
It is possible to do this in two steps:
zip archive.zip -r -# < include.lst
zip archive.zip -d -# < exclude.lst
Related
I am writing a script in Lua 5.1 for use with a game engine (EDGE).
I need my script to copy about 20 files into a .miz file (which is really a zipped folder with a set structure) and navigate that structure and copy those files in from a non-zipped folder on the hard drive.
Because Windows 11 it the future I need to use NanaZip rather than 7z which isn't W11 supported.
However, all the examples I've found are for using LUA to zip up files, not insert non-zipped files INTO a zip file without unzipping it.
Is this even possible?
Similar to #koyaanisqatsi I tried it with 7z. You didn't comment on our question on why 7z should be avoided nor whether you are even allowed to use os.execute, but it should provide a good starting point:
os.execute("7z a yourZip.zip yourFile.png")
Where a is the flag for Add.
See the manual for other flags like compression: https://linux.die.net/man/1/7z
Windows 11 also have tar that have the option r and u
D:\temp>tar h
tar(bsdtar): manipulate archive files
First option must be a mode specifier:
-c Create -r Add/Replace -t List -u Update -x Extract
Common Options:
-b # Use # 512-byte records per I/O block
-f <filename> Location of archive (default \\.\tape0)
-v Verbose
-w Interactive
Create: tar -c [options] [<file> | <dir> | #<archive> | -C <dir> ]
<file>, <dir> add these items to archive
-z, -j, -J, --lzma Compress archive with gzip/bzip2/xz/lzma
--format {ustar|pax|cpio|shar} Select archive format
--exclude <pattern> Skip files that match pattern
-C <dir> Change to <dir> before processing remaining files
#<archive> Add entries from <archive> to output
List: tar -t [options] [<patterns>]
<patterns> If specified, list only entries that match
Extract: tar -x [options] [<patterns>]
<patterns> If specified, extract only entries that match
-k Keep (don't overwrite) existing files
-m Don't restore modification times
-O Write entries to stdout, don't restore to disk
-p Restore permissions (including ACLs, owner, file flags)
bsdtar 3.5.2 - libarchive 3.5.2 zlib/1.2.5.f-ipp bz2lib/1.0.6
( Above cmd.exe was opened from Lua with: os.execute('cmd') )
You can extract a ZIP with it but not creating one as far as i know.
(tar -xf archive.zip)
But is it a Problem for you to use TAR instead of ZIP?
I am trying to get the following done. I have circa 40 directories of different species, each with 100s of sequence files that contain orthologous sequences. The sequence files are similarly named for each of the species directories. I want to concatenate the identically named files of the 40 species directories into a single sequence file which is named similarly.
My data looks as follows, e.g.:
directories: Species1 Species2 Species3
Within directory (similar for all): sequenceA.fasta sequenceB.fasta sequenceC.fasta
I want to get single files named: sequenceA.fasta sequenceB.fasta sequenceC.fasta
where the content of the different files from the different species is concatenated.
I tried to solve this with a loop (but this never ends well with me!):
ls . | while read FILE; do cat ./*/"$FILE" >> ./final/"$FILE"; done
This resulted in empty files and errors. I did try to find a solution elsewhere, e.g.: (https://www.unix.com/unix-for-dummies-questions-and-answers/249952-cat-multiple-files-according-file-name.html, https://unix.stackexchange.com/questions/424204/how-to-combine-multiple-files-with-similar-names-in-different-folders-by-using-u) but I have been unable to edit them to my case.
Could anyone give me some help here? Thanks!
In a root directory where your species directories reside, you should run the following:
$ mkdir output
$ find Species* -type f -name "*.fasta" -exec sh -c 'cat {} >> output/`basename {}`' \;
It traverses all the files recursively and merges the contents of files with identical basename into one under output directory.
EDIT: even though this was an accepted answer, in a comment the OP mentioned that the real directories don't match a common pattern Species* as shown in the original question. In this case you can use this:
$ find -type f -not -path "./output/*" -name "*.fasta" -exec sh -c 'cat {} >> output/`basename {}`' \;
This way, we don't specify the search pattern but rather explicitly omit output directory to avoid duplicates of already processed data.
I have in a folder, pairs of files, mp4 and srt subtitle files.
e.g.
video1.mp4
video1.srt
video2.mp4
video2.srt
I want to prepend a random number to each video file but I must also prepend the same number to the set file.
Does anyone know how I can do this?
I'm using a mac, so it could be a bash script, automator, or even PowerShell.
I worked it out. Here's a single line bash command that does what I need.
for f in *.mp4 *.m4v; do ran=$(printf "%03.0f" $((RANDOM%300))); myFile="${f}"; mv -f "${myFile}" "${ran}-${myFile}"; mv -f "${myFile%.*}.srt" "${ran}-${myFile%.*}.srt"; done
I want to create a windows batch file (Win7) to achieve the following:
Copy source.doc to destination with destinationFilename.doc taken from a list in a text file (nameList.txt)
I have batch file that will make directories from nameList.txt but I can't figure out how to modify the batch file to make it copy source.doc in the required manner.
Using xargs you can process a file nameList.txt which contains a newline separated list of target filenames like this:
cat nameList.txt | xargs -I "F" cp source.doc F
where -I "F" defines F as a placeholder to be used in command invocation of cp.
Using Terminal and Shell/Bash commands is there a way to retrive specific files from a web directory? I.e.
Directory: www.site.com/samples/
copy all files ending in ".h" into a folder
The folder contains text files, and other files associated that are of no use.
Thanks :)
There are multiple ways of achieving this recursively:
1. using find
1.1 making directorys using find -p to create recursive folders without errors
cd path;
mkdir backup
find www.site.com/samples/ -type d -exec mkdir -p {} backup/{} \;
1.2 finding specific files and copying to backup folder -p to perserve permissions
find www.site.com/samples/ -name \*.h -exec cp -p {} backup/{} \;
Using tar well actually for reverse type of work i.e. to exclude specific files which the part of the question related to text files matches this answer more:
You can have as many excludes as you liked added on
tar --exclude=*.txt --exclude=*.filetype2 --exclude=*.filetype3 -cvzf site-backup.tar.gz www.site.com
mv www.site.com www.site.com.1
tar -xvzf site-backup.tar.gz
You can use the wget for that, but if there are no links to that files. I.e. they exist, but they are not referenced from any html page, then bruteforce is the only option.
cp -aiv /www.site.com/samples/*.h /somefolder/
http://linux.die.net/man/1/cp