Copy-Item with exclude files and folders in powershell - arrays

I have a question. I want to copy content of one directory into another and made some progress. But, at the moment I'm stuck because if I can define array of files to exclude I have a problem with array of folders. So the arrays looks like:
$excludeFiles = #('app.config', 'file.exe')
$excludeFolders = #('Logs', 'Reports', 'Backup', 'Output')
It works when there is only one item in $excludeFolders array, but if I add multiple items, it copies all folders without excluding them.
Script I have:
Get-ChildItem -Path $binSolutionFolder -Recurse -Exclude $excludeFiles |
where { $excludeFolders -eq $null -or $_.FullName.Replace($binSolutionFolder, "") -notmatch $excludeFolders } |
Copy-Item -Destination {
if ($_.PSIsContainer) {
Join-Path $deployFolderDir $_.Parent.FullName.Substring($binSolutionFolder.length -1)
}
else {
Join-Path $deployFolderDir $_.FullName.Substring($binSolutionFolder.length -1)
}
} -Force -Exclude $excludeFiles
Where $binSolutionFolder is source and $deployFolderDir is target.
Files work fine, but with folders I've run out of ideas.

-notmatch uses a regex-pattern and not a collection. To match against a collection of words you could use -notin $excludedfolders, but if the path includes 2+ level of folders or just a simple \ then the test would fail.
I would use -notmatch, but first create a regex-pattern that checks all folders at ones. Ex:
$excludeFiles = #('app.config', 'file.exe')
$excludeFolders = #('Logs', 'Reports', 'Backup', 'Output','Desktop')
$excludeFoldersRegex = $excludeFolders -join '|'
Get-ChildItem -Path $binSolutionFolder -Recurse -Exclude $excludeFiles |
where { $_.FullName.Replace($binSolutionFolder, "") -notmatch $excludeFoldersRegex } |
.....

Related

Loop though network share and list all users folders and create/delete file

We have 1 main network share named the below, after the underscore there is 8 locations which are "abc$", "def$", "ghi$", "jkl$", "mno$", "pqr$", "stu$", "vwxyz$".
Depending on the users first letter in their username a folder is created with the which is the users home area.
Example username - AdamB will be put in Networkshare1_abc$.
Example 2 username - EdwardB will be put in Networkshare1_def$
Within this folder are the is a list of all users as a folder
Networkshare1_abc$
Networkshare1_def$
Networkshare1_ghi$
Networkshare1_jkl$
Networkshare1_mno$
Networkshare1_pqr$
Networkshare1_stu$
Networkshare1_vwxy$
I need a script that will
that will loop though only the top folder list the users
check to see if a text file exists and if it does delete it (have this working below)
$Wantedfile = "Networkshare1_stu$\user1\test.txt"
$timeStamp = (Get-Date -Format "dd-MM-yyyy-hh-mm")
$FileName = $timeStamp + ".txt"
if ((Test-Path $Wantedfile$FileName) -eq $false) {
Write-Host "file does not exists"
} elseif ((Test-Path $Wantedfile$FileName) -eq $true) {
Write-Host "file present..removing file"
Remove-Item $Wantedfile$FileName
}
The bit I am struggling with is the loop and how to get it to check each folder in the above locations as streamline as possible.
I am not sure if your files are really named like
"Networkshare1_stu$\user1\test.txt03-21-2017-19-41.txt"
but basically what you need is foreach loop. You need to get all the folders in the top folder. In $directories you will have abc$, def$ etc. folders. Then you will loop through each of them and search for the file using Where-Object.
$nameOfFile = "test.txt"
$fullPathFile = "\\Networkshare1_abc$\user1\test.txt"
$directories = Get-ChildItem "\\Networkshare1" -Directory
foreach($directory in $directories)
{
Write-Host "Searching in $directory.FullName ..."
$files = Get-ChildItem $directory -Recurse | Where-Object{$_.FullName -like "*$nameOfFile"}
# or to use fullPathFile
# $files = Get-ChildItem $directory -Recurse | Where-Object{$_.FullName -eq $fullPathFile}
foreach($file in $files)
{
Write-Host "Removing $file.FullName - continue?"
Read-Host ""
Remove-Item $file -Force
}
}

Comparing files within folders and moving files not found in one folder to a different location

I am working on folders that contain many types and sizes of files within them. What I want to do is move files that are not contained in one folder into a new folder. I have embedded a picture link that helps illustrate what I am aiming to do.
I would like test123.pdf to be moved to a new location because it's not contained within the other folder. Below I have some code that simply compares the contents of each folder and outputs which file is out of place. I have been researching some things online, but have come up empty. Can anyone help me proceed?
Disclaimer: I know the path is wrong, but I can't show it for security reasons.
$Folder1 = Get-ChildItem -Recurse -path "Enter Path here"
$Folder2 = Get-ChildItem -Recurse -path "Enter the Path here"
Compare-Object -ReferenceObject $Folder1 -DifferenceObject $Folder2
It sounds like you want to compare the contents of two folders, ID the file(s) that are not present in both folders, and then move them to a third folder. To accomplish this, you can define your 2 paths in variables, compare the contents of the folders, grab the full path names of the different items, and then move them to the new destination.
$path1 = yourfirstpath
$path2 = yoursecondpathforcomparing
$path3 = yourdestinationpath
diff (ls $path1 -recurse) (ls $path2 -recurse) | ForEach {$_.InputObject.FullName} | Move-Item -Destination $path3
diff = Compare-Object , ls = Get-ChildItem
This will do the work, any file that is not in both Folder1 & Folder2 will be moved to Folder 3
$Folder1 = 'C:\folder1'
$Folder2 = 'C:\folder2'
$Folder3 = 'C:\folder3'
foreach ($file in Get-ChildItem -Recurse $Folder1)
{
if (-not(Test-Path "$Folder2\$file"))
{
Move-Item $file.FullName $Folder3
}
}
This will compare by file name, and maintain the directory structure the extra files were found in, and handle the recursion properly. The output of the code demonstrates the directory structure I tested with.
# be explict here to ensure everything is lowercase
# so when this is cut and pasted, it does not break
# when you enter the real path
$d1 = "d:\test\one".ToLower()
$d2 = "d:\test\two".ToLower()
$f1 = gci -Recurse $d1 -File | % {$_.FullName.ToLower()}
$f2 = gci -Recurse $d2 -File | % {$_.FullName.ToLower()}
"f1"
$f1
"`nf2"
$f2
$same = #()
$extra = #()
foreach ($f in $f2)
{
$f2tof1path = $f.Replace($d2, $d1)
if ($f1.Contains($f2tof1path) -eq $false)
{
$extra += $f
}
else {
$same += $f
}
}
"`nSame"
$same
"`nExtra"
$extra
"`nMOVE"
$folder3 = "d:\test\three"
foreach ($f in $extra)
{
# move files somewhere, create dir if not exists
$dest = $f.Replace($d2,$folder3)
$destdir = $(Split-Path -Parent -Path $dest)
if (!(Test-Path $destdir))
{
# remove quotes to do new-item, keep them to show what it will do
"New-Item -ItemType Directory -Force -Path $destdir"
}
# remove quotes to do move-item, keep them to show what it will do
"Move-Item $f $dest"
}
Output
f1
d:\test\one\1.txt
d:\test\one\2.txt
d:\test\one\sub\1.txt
d:\test\one\sub\2.txt
f2
d:\test\two\1.txt
d:\test\two\2.txt
d:\test\two\3.txt
d:\test\two\sub\1.txt
d:\test\two\sub\2.txt
d:\test\two\sub\3.txt
Same
d:\test\two\1.txt
d:\test\two\2.txt
d:\test\two\sub\1.txt
d:\test\two\sub\2.txt
Extra
d:\test\two\3.txt
d:\test\two\sub\3.txt
MOVE
New-Item -ItemType Directory -Force -Path d:\test\three
Move-Item d:\test\two\3.txt d:\test\three\3.txt
New-Item -ItemType Directory -Force -Path d:\test\three\sub
Move-Item d:\test\two\sub\3.txt d:\test\three\sub\3.txt
If this does not solve your problem, it should get you 99% there. Play around with the code, have fun, and good luck.

Zipping older files with powershell

I have the below Powershell script which tries to archive the logs.
1st step is to move all the files that contain Spotfire.Dxp...* string the ProjectLogsDir directory.
2nd step is identify the Spotfire.Dxp...* files in RotatedLogsDir directory that are older than 60 days and put them in a zip file together in a ArchivedLogsDir directory
3rd step is to delete zipped file older than 120 days.
here 2nd step function isn't working with error
E:\TIBCO\logsArchival\rotatedDir\C0005749_2014-05-09-12-53-47_Spotfire.Dxp.Web.231.log E:\TIBCO\logsArchival\rotatedDir\C0005749_2014-05-09-12-53-47_Spotfire.Dxp.Web.232.log: WARNING: The filename, directory name, or volume label syntax is incorrect.
$sysname=$env:COMPUTERNAME
$Date = Get-Date
$Now = Get-Date -format yyyy-MM-dd-HH-mm-ss
$host_date=$sysname +"_"+ $Now
$RotateDays = "60"
$ArchiveDays="120"
$ProjectLogsDir = "E:\TIBCO\*\6.0.0\Logfiles"
$RotatedLogsDir = "E:\TIBCO\logsArchival\rotatedDir"
$ArchivedLogsDir= "E:\TIBCO\logsArchival\archiveDir"
$psLogsDir= "E:\TIBCO\logsArchival\shLogsDir"
$LastRotate = $Date.AddDays(-$RotateDays)
$LastArchive = $Date.AddDays(-$ArchiveDays)
$RenameLogFiles = Get-Childitem $ProjectLogsDir -include Spotfire.Dxp.*.*.* -exclude spotfire.Dxp.web.KeepAlive.* -Recurse
$RenameLogFiles
$RenameLogFiles | Where-Object {!$_.PSIsContainer} | Rename-Item -NewName { $host_date +"_"+ $_.Name.Replace(' ','_') };
$RotatedLogFiles = Get-Childitem $ProjectLogsDir -include *_Spotfire.Dxp.*.*.* -Recurse
$RotatedLogFiles
$RotatedLogFiles | move-item -destination "$RotatedLogsDir"
$ZippedLogFiles = Get-Childitem $RotatedLogsDir -include *_Spotfire.Dxp.*.*.* -Recurse | Where {$_.LastWriteTime -le "$LastRotate"}
$ZippedLogFiles
function create-7zip([String] $aDirectory, [String] $aZipfile) {
[string]$pathToZipExe = "C:\Program Files\7-zip\7z.exe";
[Array]$arguments = "a", "-tzip", "$aZipfile", "$aDirectory";
& $pathToZipExe $arguments;
}
create-7zip "$ZippedLogFiles" "$ArchivedLogsDir\$host_date.zip"
$ZippedLogFiles | Remove-Item -Force
$DeleteZippedFiles = Get-Childitem $ArchivedLogsDir\*.zip -Recurse | Where {$_.LastWriteTime -le "$LastArchive"}
$DeleteZippedFiles
$DeleteZippedFiles | Remove-Item -Force
$DeletePsFiles = Get-Childitem $psLogsDir\*.log -Recurse | Where {$_.LastWriteTime -le "$LastRotate"}
$DeletePsFiles
$DeletePsFiles | Remove-Item -Force
Please provide the help here to get the files zipped.
The issue is that you are calling your 7-Zip function incorrectly. Look at what the function takes:
function create-7zip([String] $aDirectory, [String] $aZipfile) {
It has 2 strings as parameters. Then look at what you are calling it with:
create-7zip "$ZippedLogFiles" "$ArchivedLogsDir\$host_date.zip"
"$ZippedLogFiles" was defined just before this by running $ZippedLogFiles = Get-Childitem $RotatedLogsDir with a few parameters to filter the results. So that right there is an array of FileInfo objects... not a string. That's the issue, is that you are not calling the function correctly.
What you really want to include is "$RotatedLogsDir\*_Spotfire.Dxp*.*" so try calling it with that instead of "$ZippedLogFiles".
Edit: Comment moved to answer for zipping only files over 60 days.
You can use built in .Net calls to archive things and not have to use 7-Zip. Even easier in my opinion would be to install the PowerShell Community Extensions and use their Write-Zip cmdlet like this:
$ZippedLogFiles = Get-Childitem $RotatedLogsDir -include *_Spotfire.Dxp.*.*.* -Recurse | Where {$_.LastWriteTime -le "$LastRotate"}
$ZippedLogFiles
$ZippedLogFiles | Write-Zip "$ArchivedLogsDir\$host_date.zip"

Comma delimited input for a powershell function

I currently have a function that I made that gets a list of all files in a directory and then adds them to an array called $FileListArray.
I want to add an option that lets me specify the file extensions to exclude from the arrary so that I could call the function as follows ListFiles -FilesToList "c:\test" –exclude “avi,txt,bmp” and this would then ignore files with any of the file extensions I have put in.
My Function so far
Function ListFiles($FilesToList){
$FileListArray = #()
Foreach($file in Get-ChildItem $FilesToList -Force -Recurse | Where-Object {$_.attributes -notlike "Directory"})
{
$FileListArray += ,#($file.name,$file.fullname,$File.Extension)
}
}
Listfiles -FilesToList "c:\tools"
Try this:
Function ListFiles($Path,$Exclude)
{
Get-ChildItem -Path $FilesToList -Force -Recurse |
Where-Object {!$_.PSIsContainer -and $_.Extension -notmatch ($Exclude -join '|') }
}
Listfiles -Pathc:\tools –Exclude avi,txt,bmp

How to delete empty subfolders with PowerShell?

I have a share that is a "junk drawer" for end-users. They are able to create folders and subfolders as they see fit. I need to implement a script to delete files created more than 31 days old.
I have that started with Powershell. I need to follow up the file deletion script by deleting subfolders that are now empty. Because of the nesting of subfolders, I need to avoid deleting a subfolder that is empty of files, but has a subfolder below it that contains a file.
For example:
FILE3a is 10 days old. FILE3 is 45 days old.
I want to clean up the structure removing files older than 30 days, and delete empty subfolders.
C:\Junk\subfolder1a\subfolder2a\FILE3a
C:\Junk\subfolder1a\subfolder2a\subfolder3a
C:\Junk\subfolder1a\subfolder2B\FILE3b
Desired result:
Delete: FILE3b, subfolder2B & subfolder3a.
Leave: subfolder1a, subfolder2a, and FILE3a.
I can recursively clean up the files. How do I clean up the subfolders without deleting subfolder1a? (The "Junk" folder will always remain.)
I would do this in two passes - deleting the old files first and then the empty dirs:
Get-ChildItem -recurse | Where {!$_.PSIsContainer -and `
$_.LastWriteTime -lt (get-date).AddDays(-31)} | Remove-Item -whatif
Get-ChildItem -recurse | Where {$_.PSIsContainer -and `
#(Get-ChildItem -Lit $_.Fullname -r | Where {!$_.PSIsContainer}).Length -eq 0} |
Remove-Item -recurse -whatif
This type of operation demos the power of nested pipelines in PowerShell which the second set of commands demonstrates. It uses a nested pipeline to recursively determine if any directory has zero files under it.
In the spirit of the first answer, here is the shortest way to delete the empty directories:
ls -recurse | where {!#(ls -force $_.fullname)} | rm -whatif
The -force flag is needed for the cases when the directories have hidden folders, like .svn
This will sort subdirectories before parent directories working around the empty nested directory problem.
dir -Directory -Recurse |
%{ $_.FullName} |
sort -Descending |
where { !#(ls -force $_) } |
rm -WhatIf
Adding on to the last one:
while (Get-ChildItem $StartingPoint -recurse | where {!#(Get-ChildItem -force $_.fullname)} | Test-Path) {
Get-ChildItem $StartingPoint -recurse | where {!#(Get-ChildItem -force $_.fullname)} | Remove-Item
}
This will make it complete where it will continue searching to remove any empty folders under the $StartingPoint
I needed some enterprise-friendly features. Here is my take.
I started with code from other answers, then added a JSON file with original folder list (including file count per folder). Removed the empty directories and log those.
https://gist.github.com/yzorg/e92c5eb60e97b1d6381b
param (
[switch]$Clear
)
# if you want to reload a previous file list
#$stat = ConvertFrom-Json (gc dir-cleanup-filecount-by-directory.json -join "`n")
if ($Clear) {
$stat = #()
} elseif ($stat.Count -ne 0 -and (-not "$($stat[0].DirPath)".StartsWith($PWD.ProviderPath))) {
Write-Warning "Path changed, clearing cached file list."
Read-Host -Prompt 'Press -Enter-'
$stat = #()
}
$lineCount = 0
if ($stat.Count -eq 0) {
$stat = gci -Recurse -Directory | %{ # -Exclude 'Visual Studio 2013' # test in 'Documents' folder
if (++$lineCount % 100 -eq 0) { Write-Warning "file count $lineCount" }
New-Object psobject -Property #{
DirPath=$_.FullName;
DirPathLength=$_.FullName.Length;
FileCount=($_ | gci -Force -File).Count;
DirCount=($_ | gci -Force -Directory).Count
}
}
$stat | ConvertTo-Json | Out-File dir-cleanup-filecount-by-directory.json -Verbose
}
$delelteListTxt = 'dir-cleanup-emptydirs-{0}-{1}.txt' -f ((date -f s) -replace '[-:]','' -replace 'T','_'),$env:USERNAME
$stat |
? FileCount -eq 0 |
sort -property #{Expression="DirPathLength";Descending=$true}, #{Expression="DirPath";Descending=$false} |
select -ExpandProperty DirPath | #-First 10 |
?{ #(gci $_ -Force).Count -eq 0 } | %{
Remove-Item $_ -Verbose # -WhatIf # uncomment to see the first pass of folders to be cleaned**
$_ | Out-File -Append -Encoding utf8 $delelteListTxt
sleep 0.1
}
# ** - The list you'll see from -WhatIf isn't a complete list because parent folders
# might also qualify after the first level is cleaned. The -WhatIf list will
# show correct breath, which is what I want to see before running the command.
To remove files older than 30 days:
get-childitem -recurse |
? {$_.GetType() -match "FileInfo"} |
?{ $_.LastWriteTime -lt [datetime]::now.adddays(-30) } |
rm -whatif
(Just remove the -whatif to actually perform.)
Follow up with:
get-childitem -recurse |
? {$_.GetType() -match "DirectoryInfo"} |
?{ $_.GetFiles().Count -eq 0 -and $_.GetDirectories().Count -eq 0 } |
rm -whatif
This worked for me.
$limit = (Get-Date).AddDays(-15)
$path = "C:\Some\Path"
Delete files older than the $limit:
Get-ChildItem -Path $path -Recurse -Force | Where-Object { !$_.PSIsContainer -and $_.CreationTime -lt $limit } | Remove-Item -Force
Delete any empty directories left behind after deleting the old files:
Get-ChildItem -Path $path -Recurse -Force | Where-Object { $_.PSIsContainer -and (Get-ChildItem -Path $_.FullName -Recurse -Force | Where-Object { !$_.PSIsContainer }) -eq $null } | Remove-Item -Force -Recurse

Resources