I want to merge the content of multiple text files to one text file.
I've tried cat explained in this answer. But it is very slow.
The copy command is much faster but you have to put the filenames in a plus sign separated string like:
cmd /c copy file1.txt + file2.txt + file3.txt + file1.txt all.txt
It is ok for a few files, but not for thousands of files.
So my idea was to create a variable that contains the file input for copy like:
%list = 'file1.txt + file2.txt + file3.txt + file1.txt'
and then:
cmd /c copy %list all.txt
But this doesn't work.
(I can create the string with the filenames also within Powershell with a loop.)
Now I want to make a loop that merges the first file with the second file and the resulting file with the third file and so on.
cmd /c copy file1.txt + file2.txt merge1.txt
then:
cmd /c copy merge1.txt + file3.txt merge2.txt
...
How can I do this within a loop in Powershell?
# Forces the creation of your content file
New-Item -ItemType File ".\all.txt" –force
# Build your file list here
$fileList = #('file1.txt', 'file2.txt', 'file3.txt')
# Assumes that all files are in the directory where you run the script
# You might have to adapt it to provide full path (e.g. $_.FullName)
$fileList | %{ Get-Content $_ -read 1000 } | Add-Content .\all.txt
Related
I did the following script in PowerShell:
(Get-Content text\text.txt).replace('ABC', 'abc') | Set-Content text\text.txt
it works as expected...
it reads the contents of a file and replaces a particular character sequence by another seguency...
but, I need that script identify multiple files with different names in others possibles subdirectories of the same initial directory.
dir01\file_name.abc
dir01\filename.def
dir01\dir02\fi-le_na-me.ghi
dir01\dor02\fi_le-na_me.jkl
dir01\dir02\dir03\f-i-l-e_n-a-m-e.mno
etc...
I do not know which will be the names of the files nor the directories too!
In msdos I use the following expression to go from file to file:
for %%F in (*.*) do ("winRAR.exe" "%%F")
this batch, compresses file by file... its only a sample!
I do not know how to identifying file by file in PowerShel... Could someone help me with that?!
Just use Get-ChildItem and pipe the output into your command. You can tell Get-ChildItem to recurse directories using -recurse and only return files with -File:
Get-ChildItem dir01 -recurse -file |
foreach{(Get-Content $_.FullName).replace('ABC', 'abc') | Set-Content $_.FullName}
I want to move few 100 pdf files to another directory from a source containing 1000s pdf. I have text file which has all the pdf file names listed in separate lines. Program has to read the file name from the text file and find that in source folder, if the file size is 12 KB and is older than 2014, Move it to destination 'b'.
Im working on windows 2008 R2.
Another option:
Batch file to extract file names from input file:
for /f %%i in (InputFile.txt) do call bat2.bat %%i
Batch file to get file size:
for %%i in (%1.txt) do echo %%~zi
...and continue from here
if you dont mind using powershell :
#variables initialisation
$filelist=c:\temp\list.txt
$source="c:\temp\pdf"
$destination="c:\temp2"
#read the content of yout list
$files=get-content $filelist
#will process the following instructions for all the filename in filelist
$files | foreach {
$f=get-childitem $source -filter "$_" -recurse #try to find the file in sourcedir
if( (($f.size /1KB) -eq 12) -and ($f.lastwritetime.year -lt 2014)){ #check if size is 12KB and last modification time occured before 2014
move-item $_.fullname $destination #move the file
}
}
I have found a few similar problems on Stackoverflow, and after trying out the solutions, my problem remains unsolved.
Here is my directory structure:
D:\XYZ\Adam Jay\Adam Jay\files.txt
D:\XYZ\Adam Jay\Adam Jay\SomeFolder\
D:\XYZ\Adam Jay\Adam Jay\OtherFolder\Some File.doc
D:\XYZ\Mary Poppins\Mary Poppins\myOtherFile.txt
and I have about 2000 of these.
My goal is to simply cut out the redundant sub-sub folder, following the above structure. Manually, I would simply cut the "sub-sub" folder "Adam Jay" and paste it into XYZ, where-in it will replace or merge with the upper-level "Adam Jay" (moving whatever files and folders with it).
The desired result:
D:\XYZ\Adam Jay\files.txt
D:\XYZ\Adam Jay\SomeFolder\
D:\XYZ\Adam Jay\OtherFolder\Some File.doc
D:\XYZ\Mary Poppins\myOtherFile.txt
I have very little knowledge of batch scripts, and no knowledge of PowerShell. By modifying a script I found here on StackOverflow, I managed to mess things up (luckily, it was a test directory). [The script I played with can be found here:
PowerShell Script to move folders one level up and delete the previous containing folder ]
It would be great if anyone can help me out with this. I would really appreciate it.
Thanks.
This should do it:
#echo off
setlocal enabledelayedexpansion
cd /d D:\XYZ
for /f "tokens=*" %%a in ('dir /b /ad') do (
if exist "%%a\%%a" (
xcopy /E "%%a\%%a\*" "%%a"
rd /s /q "%%a\%%a"
)
)
I'll go the PowerShell route. This has two options, by default it will remove any double folders so...
Z:\XYZ\John Doe\John Doe\My Documents\Stuff\Stuff\ItsAFile.txt
becomes:
Z:\XYZ\John Doe\My Documents\Stuff\ItsAFile.txt
If you only want to get rid of the first duplicate folder comment out the 6th line, and uncomment the 7th. If you don't like the reporting part remove/comment out the Write-Output lines.
$BasePath = "D:\XYZ"
gci $BasePath -recurse | %{
$OriginalFile = $_
If($OriginalFile.PSIsContainer){$FilePath = $_.FullName.ToString().Split('\')}
Else{$FilePath = $_.Directory.ToString().Split('\')}
for($i=1;$i -lt $FilePath.Count;$i++){ #Comment out this line to change to Base Path +2 folders method
# for($i=0;$i -lt $($BasePath.Split('\').Count +2);$i++){ #Uncomment to only affect the first 2 folders past the base path
if($filepath[$i] -and ($FilePath[$i] -ieq $FilePath[$i-1])){
$FilePath[$i-1] = $Null
}
}
($FilePath|?{$_}) -join '\'|%{
if($OriginalFile.PSIsContainer){
If(!($OriginalFile.FullName -ieq $_)){
Write-Output "Moving folder $($OriginalFile.fullname) to $_"
move-item "$($OriginalFile.fullname)\*" $_ -ErrorAction SilentlyContinue
Remove-Item $OriginalFile.fullname -ErrorAction SilentlyContinue}
}else{
If(!($OriginalFile.Directory.ToString() -ieq $_)){
Write-Output "Moving file $($OriginalFile.fullname) to $(Resolve-Path "$_\..")"
move-item $OriginalFile.fullname "$_\.." -ErrorAction SilentlyContinue
}}
}
}
To step through that, it:
Pulls a directory listing of each folder or file in the base path.
For each one it splits up the path based on the backslash character,
excluding file names.
It then steps through each segment comparing each folder to the one
before it in the path (or only steps the first few segments if you
go that route).
If the two sequential folders match it removes the first one.
It then rebuilds the path at the end of that process.
After that it compares the modified path to the original, and if
they are different it moves that item to the modified path.
It does handle files and folders differently, as it uses the .FullName property for folders, and the .Directory property for files.
I could really use some help. I have searched the topics here and on the web in general and got closer to what I need but not the whole way
I need a list of all the files and sub folders within a folder
I have found a lot of help in how to get the files and folders from a directory using Command Prompt, but in my case the directory is massive and would rather just specify the folder I need and then get the files that exist within that folder (and the sub folders and their files). Please also note I am not very technical and can just about manage basic queries with Command Prompt
I need:
A list that I can import into excel that contains - File name, File Path if possible, Amended Date, Author, File Type
The files I need all exist within subfolders in M:\Sales & Marketing\Sales\Sales Tools and some more that I need in M:\Sales & Marketing\Sales\Tenders\Tender Docs
I thought of copying them from the shared folder into my local directory but that was going to take over 2 hours to do
I really would appreciate any help that you can offer. I have tried changing all the examples along the lines of dir /s /b >> filelist.txt to use the folder and path but no luck
Many many thanks in advance
Save all this into a batch file called say MyInfo.bat and then launch it. In the same folder you should find file.csv with some info inside it that may help you. It uses Powershell to get the information but can be launched as a batch file.
Where you mention Author, this returns the owner of the file. The file type is shown by the extension in the filename.
#echo off
more +7 "%~f0" >"%temp%\a.ps1"
powershell "%temp%\a.ps1" >"file.csv"
del "%temp%\a.ps1"
goto :EOF
$files = Get-ChildItem 'M:\Sales & Marketing\Sales\Sales Tools\*' -Recurse
foreach ($file in $files){
$acl = $file | get-acl | select path,owner
$result = "`"$($file.name.ToString())`",`"$($file.directoryname.ToString())`",`"$($acl.owner.ToString())`",$($file.lastwritetime.ToString("yyyy-MM-dd hh:mm:ss"))"
write-output $result
}
$files = Get-ChildItem 'M:\Sales & Marketing\Sales\Tenders\Tender Docs\*' -Recurse
foreach ($file in $files){
$acl = $file | get-acl | select path,owner
$result = "`"$($file.name.ToString())`",`"$($file.directoryname.ToString())`",`"$($acl.owner.ToString())`",$($file.lastwritetime.ToString("yyyy-MM-dd hh:mm:ss"))"
write-output $result
}
Here is some code I made :)
#echo off
set source="R:\Contracts\"
set destination="R:\Contracts\Sites\"
ROBOCOPY %source% %destination% *.srt *.pdf *.mp4 *.jpg /COPYALL /R:0 /S
for /r %source in (*) do #copy "%destination" .
R:\Contracts\ is full of folders which have files in them.
I want to copy all to R:\Contracts\Sites\ and flatten the folder structure.
Everything copies well but also the folder structure.
Thank you
You could do that with a PowerShell one liner.
In this example I filter out all the files with the .txt extension from all the subfolders. And then send them to the Copy-Item Cmdlet.
Combine the Cmdlets Get-Childitem (GCI for short), -recurse, and -filter and then pipe the result to the Copy-Item Cmdlet. Use -WhatIf first to check that the output is what you expected.
Copy to another folder (Use -WhatIf and verify the output to check your command before committing to copying the files):
Get-Childitem -recurse R:\Contracts -filter *.txt | Copy-Item -Destination R:\Contracts\Sites -WhatIf
To do multiple filetypes as you've asked, you can just run multiple commands, one for each filetype.
No single command will flatten the hierarchy for you; you will have to use multiple commands. It can be done simply by using FOR /R to walk the hierarchy, coupled with your copy/move command of choice (move, copy, xcopy, robocopy). Because your destination is within the source hierarchy, you need an IF to prevent the destination from being a source.
Before proceeding you should stop and think about what happens if the same file name appears in multiple source folders. You can only have one version in your destination folder. Can you guarantee no duplicate names exist? If not, which file should be kept? How can you structure the command to keep the file you want? This complication is probably why no command was ever written to simply flatten a hierarchy.
Here is your ROBOCOPY command integrated with the FOR /R solution.
#echo off
set source="R:\Contracts"
set destination="R:\Contracts\Sites"
::Not sure if this is needed
::It guarantees you have a canonical path (standard form)
for %%F in (%destination%) do set destination="%%~fF"
for /r %source% %%F in (.) do if "%%~fF" neq %destination% ROBOCOPY "%%F" %destination% *.srt *.pdf *.mp4 *.jpg /COPYALL /R:0
Similar to the previous Powershell option, I did the following to flatten a multi-subdirectory music folder:
#Get all files and not the directories
$files = Get-ChildItem -Path R:\Contracts -Recurse | Where {$_.PSIsContainer -eq $false}
#Copy items from sources to new destination
foreach ($file in $files){
Copy-Item -Path $file.FullName -Destination R:\Contracts\Sites\$($file.Name)
}
The Get-ChildItem with the -Recurse switch will get a listing of all sub-folders and files. The Where function is stripping out any directories by checking the boolean PSIsContainer property. Without stripping the directories the sub-folder structure would be created without files in them. This listing is stored in the $files variable.
The foreach function runs through the list of files in the $files variable and stores one item at a time in the $file variable. The Copy-Item function then uses the full path from $file.FullName and then copies the file to the destination with the same name from $file.Name.
I recently had to tackle this problem, and many files that I wanted to move to from the hierarchy to a single folder had the same name as each other, and I wanted to still flatten the hierarchy without them to being over-written.
What I did was write a script that moves the file, but renames it with the old hierarchy path in the name
for example:
source files:
C:\files\somefiles\file.txt
C:\files\otherfiles\file.txt
destination is C:\newdir\
files are created as
C:\newdir\somefiles-file.txt
C:\newdir\otherfiles-file.txt
here is the code, batch file 1 goes thru the files, batch file 2 renames and moves them (could also copy instead, if you want to preserve the source:
#echo off
for /r %%f in (*.*pr) do #renameandmovefilespart2.bat "%%f" "%%~ff" "%%~xf"
renameandmovefilespart2.bat
#echo off
Setlocal EnableDelayedExpansion
rem set the whole file path
set origWhole=%1
set origPathOnly=%2
set extension=%3
rem here you can set where the directory to hold the flattened hierarchy is
set destDir=c:\destinationDir\
rem set the directory to do a string replace
rem make this the starting directory, that you dont want in the newly renamed files
set startingDir=C:\starting\directory\
set nothing=
set slash=\
rem here you can set what the character to represent the directory indicator \ in the new files
set reaplcementDirectoryCharacter=--
set quote="
rem cut out the starting part of the directory
call set newname=%%origWhole:!startingDir!=!nothing!%%
rem replace slashes with new character
call set newname=%%newname:!slash!=!reaplcementDirectoryCharacter!%%
rem remove quotes
call set newname=%%newname:!quote!=!nothing!%%
rem #echo shortened: %newname%
rem #echo source path: %origPathOnly% newPath: %startingDir%
rem #echo extension: %extension%
rem rename the files
ren %origWhole% %newname%
rem prepare to move the file, clean up the source path
call set origPathOnly=%%origPathOnly:!quote!=!nothing!%%
move "%origPathOnly%%newname%" "%destDir%"
There's an old PCMag utility called Sweep.exe that can operate the same command in the current and subdirectory. I wouldn't put the destination as a subdirectory of the source directory. Put the destination elsewhere.
http://www.rarewares.org/files/case/Sweep.zip
cd c:\contracts
sweep copy *.* c:\sites
This will copy everything from c:\contracts and underneath to c:\sites
I use a similar command to flatten the hierarchy of a directory.
Take care with duplicates and how you want to handle them. Do you want to overwrite or handle a different way.
For those familiar with R copying files to a single folder could be easily scripted as follow. It is more efficient than ROBOCOPY for large sets and directory structures.
## R script - Flatten files in folder structure
# declare your paths
sourcepath<-C:\\Myfolder\\PicturesInStructure"
destpath<-"C:\\Myfolder\\PicturesTogether\\"
# read source filenames with path to vector variable
fileslist<-list.files(sourcepath,
pattern="*.jpg", # in this example copies JPG pictures
full.names=TRUE, # retrieves full path
recursive=TRUE, # runs through all subdirs
include.dirs=FALSE) # skips dir names (no to be copied)
# here you will get a head of your list
head(fileslist)
# ordinary loop by source filenames
for (i in 1:length(fileslist)) {
# Number of Slashes required to extract filename from the source path
NoS<-lengths(regmatches(fileslist[i], gregexpr("/", fileslist[i])))
# for lengthy processes it is good to know where you are with this
cat(i,fileslist[i]," \r")
# copy one file to a destination folder (full path composed with paste0())
file.copy(fileslist[i],paste0(destpath,unlist(strsplit(fileslist[i],"/"))[NoS+1]))
}