I want to compare and sync two directories in two different locations. Using the diffs, I want to copy files from directory 1 to directory 2, and directory 2 to directory 1 (assuming each directory has files the other doesn't). I'll explain the issue with an example.
dir1: c:\john\abc
dir2: a:\mary\abc
let's say has 10 files. 1 diff.
c:\john\abc\folder\file.txt
a:\mary\abc\file.txt
If I do compare object dir1 dir2 -property name. This 1 diff doesn't show up because I'm only comparing file names. If I do compare-object dir1 dir2 property fullname then everything is a diff, so that won't work.
ok, I can get around that. I chop off the root dir path, so that I compare these:
folder\file.txt
file.txt
(2 diffs found)
Issue: How do I know which directory has the file.txt inside a folder, and which directory has file.txt in the root? (c:\john\abc vs a:\mary\abc) ?
Related
I would like to move file by file from one folder to another starting with the last file and so on until there are no more files in the source folder.
I already managed to find and isolate the last file in the source folder with find and move to directory folder with mv:
find ~/Music/Music -not -path '*/\.*' -type f | gtac | head-1 | xargs -I '{}' mv {} ~/Music/iTunes/iTunes\ Media/Automatically\ Add\ to\ Music.localized
Is it also possible to add a rest time between the different mv?
Thank you in advance for your answers
I am trying to get the following done. I have circa 40 directories of different species, each with 100s of sequence files that contain orthologous sequences. The sequence files are similarly named for each of the species directories. I want to concatenate the identically named files of the 40 species directories into a single sequence file which is named similarly.
My data looks as follows, e.g.:
directories: Species1 Species2 Species3
Within directory (similar for all): sequenceA.fasta sequenceB.fasta sequenceC.fasta
I want to get single files named: sequenceA.fasta sequenceB.fasta sequenceC.fasta
where the content of the different files from the different species is concatenated.
I tried to solve this with a loop (but this never ends well with me!):
ls . | while read FILE; do cat ./*/"$FILE" >> ./final/"$FILE"; done
This resulted in empty files and errors. I did try to find a solution elsewhere, e.g.: (https://www.unix.com/unix-for-dummies-questions-and-answers/249952-cat-multiple-files-according-file-name.html, https://unix.stackexchange.com/questions/424204/how-to-combine-multiple-files-with-similar-names-in-different-folders-by-using-u) but I have been unable to edit them to my case.
Could anyone give me some help here? Thanks!
In a root directory where your species directories reside, you should run the following:
$ mkdir output
$ find Species* -type f -name "*.fasta" -exec sh -c 'cat {} >> output/`basename {}`' \;
It traverses all the files recursively and merges the contents of files with identical basename into one under output directory.
EDIT: even though this was an accepted answer, in a comment the OP mentioned that the real directories don't match a common pattern Species* as shown in the original question. In this case you can use this:
$ find -type f -not -path "./output/*" -name "*.fasta" -exec sh -c 'cat {} >> output/`basename {}`' \;
This way, we don't specify the search pattern but rather explicitly omit output directory to avoid duplicates of already processed data.
So, I'm completely new to scripting and batch files.
I'm looking to copy files based on a list of the files to copy. I have a text file for the list of files to copy and I want the script to copy all these files line by line from a source directory.
For example I have over a 1000 files to copy, and there are 3 files for each folder I want to create and put them into. Below is an example of how the files names are formatted:
file3_AB12_autoc.pdf
file3_AB12.jpeg
file1_AB12.png
file3_CD34_autoc.pdf
file3_CD34.jpeg
file1_CD34.png
...etc...
Once these are copied, I want to move them into folders that I created using a text file and the command:
FOR /F %i in (folders_list.txt) do md %i
Now, with my script, I want to move the files it copies into their corresponding folders. Basically if the file name contains the folder name, then they should move into the folder. For example:
C:\AB12 THIS FOLDER SHOULD HAVE ALL THE FOLLOWING FILES
file3_AB12.jpeg
file3_AB12_autoc.pdf
file1_AB12.png
C:\CD34 THIS FOLDER SHOULD HAVE ALL THE FOLLOWING FILES
file3_CD34.jpeg
file3_CD34_autoc.pdf
file1_CD34.png
I have looked up other similar questions but nothing seems to work for what I want to do. I believe the links below are useful but I do not know how to put them together.
How do I copy files into folders based on the file containing the folder name?
Copy files based on a list stored in .txt file
In PowerShell, Move-Item will move the files. Then you just need regular expressions to determine the path and new file name, similar to:
$txt = "file3_AB12_autoc.pdf
file3_AB12.jpeg
file1_AB12.png
file3_CD34_autoc.pdf
file3_CD34.jpeg
file1_CD34.png
badfilename.txt"
foreach ($orig_file in $txt -split '\r\n') {
if ( $orig_file -match '^([^_]*)_([^_.]*)(.*)\.(.*)$' ) {
$dir = $matches[2]
$new_file = '{0}{1}.{2}' -f $matches[1], $matches[3], $matches[4]
'Moving {0} to {1}\{2}' -f $orig_file, $dir, $new_file
Move-Item -Path $orig_file -Destination ('{0}\{1}' -f $dir, $new_file)
}
else {
'{0} not processed' -f $orig_file
}
}
I am looking for a script to rename files and directories that have special characters in them.
My files:
?rip?ev <- Directory
- Juhendid ?rip?evaks.doc <- Document
- ?rip?ev 2 <- Subdirectory
-- t?ts?.xml <- Subdirectory file
They need to be like this:
ripev <- Directory
- Juhendid ripevaks.doc <- Document
- ripev 2 <- Subdirectory
-- tts.xml <- Subdirectory file
I need to change the files and the folders so that the filetype stays the same as it is for example .doc and .xml wont be lost. Last time I did it with rename it lost every filetype and the files were moved to mother directory in this case ?rip?ev directory and subdirectories were empty. Everything was located under the mother directory /home/samba/.
So in this case I need just to rename the question mark in the file name and directory name, but not to move it anywhere else or lose any other character or the filetype. I have been looking around google for a answer but haven't found one. I know it can be done with find and rename, but haven't been able to over come the complexity of the script. Can anyone help me please?
You can just do something like this
find -name '*\?*' -exec bash -c 'echo mv -iv "$0" "${0//\?/}"' {} \;
Note the echo before the mv so you can see what it does before actually changing anything. Otherwise above:
searches for ? in the name (? is equivalent to a single char version of * so needs to be escaped)
executes a bash command passing the {} as the first argument (since there is no script name it's $0 instead of $1)
${0//\?/} performs parameter expansion in bash replacing all occurrences of ? with nothing.
Note also that file types do not depend on the name in linux, but this should not change any file extension unless they contain ?.
Also this will rename symlinks as well if they contain ? (not clear whether or not that was expected from question).
I usually do this kind of thing in Perl:
#!/usr/bin/perl
sub procdir {
chdir #_[0];
for (<*>) {
my $oldname = $_;
rename($oldname, $_) if s/\?//g;
procdir($_) if -d;
}
chdir "..";
}
procdir("top_directory");
I'm trying to write a script, that will select N number of files randomly, and move them to folders named 1, 2, 3.. Then it must compress each folder, and run another batch/script.
The script I use to randomly copy N number of files, is:
gci somefolder | random -c $x | mi -dest someotherfolder
Example of what I need:
C:\somefolder (has 11000 files)
C:\1 (has 2000 files)
C:\2 (has 2000 files)
C:\3 (has 2000 files)
C:\4 (has 2000 files)
C:\5 (has 2000 files)
C:\6 (has 1000 files)
Then each folder is compressed to
C:\1.zip
C:\2.zip
C:\3.zip
C:\4.zip
C:\5.zip
C:\6.zip
And lastly, it will run a simple another batch file (Thinking of FTP'ing files)
A big thank you to //\O// for his out-zip function! This will do what you want:
function out-zip {
Param([string]$path)
if (-not $path.EndsWith('.zip')) {$path += '.zip'}
if (-not (test-path $path)) {
set-content $path ("PK" + [char]5 + [char]6 + ("$([char]0)" * 18))
}
$ZipFile = (new-object -com shell.application).NameSpace($path)
$input | foreach {$zipfile.CopyHere($_.fullname)}
}
$Path = "C:\Temp"
$FileList = gci $path
[int]$FilesPerZip=100
For($i=1;$i -le [Math]::Ceiling($FileList.count/$FilesPerZip);$i++){
md "c:\$($i)"
gci $path|random -Count $FilesPerZip|%{$_.moveto("c:\$i\$($_.name)")}
}
(1..[Math]::Ceiling($FileList.count/$FilesPerZip))|%{gi "c:\$_"|out-zip "C:\$_.zip"}
Just update your path and how many files you want in each zip file.
This counts how many files in total, divides by how many files you want per zip file, and rounds up. Then it loops through the target directory grabbing X files at random and placing them into sequential numbered folders. Lastly it zips those folders.
It will fail if there are already zip files with the desired name existing already (i.e. C:\1.zip), and will probably throw errors if there are those folders there as well, and will fail if there are those folders AND there are matching files in them, so you will probably want to throw in some checks up front to make sure those folders and files don't exist already, but this answers your Move & Zip question.