Powershell array failing because of "Copy-Item : Illegal characters in path." - arrays

I have a specific usecase where I need to identify if files from a list exist, and if so, copy them to a separate location with the relevant file structure kept. I need to keep my list of targets in the same script.
I believe my issue is something to do with the way the data inside isn't being parsed correctly due to ":" for drive letters, but I'm unsure of how to get round this issue.
As you can see from the code below, I attempted to fix the issue by ignoring the drive letter, and appending it during the Copy-Item, but it doesn't seem to work either. (e.g: C:\folder\file becomes \folder\file in the list.)
I created test directory to just help show the issue, of examples of files/folders that I want to grab (purely for testing, the real files are multiple locations/file types).
- test_dir_cmd
- folder
- folder1
* file2.db
* file3.json
* file2.txt
* file3.js
- folder2
* file.bak
* file.db
* file.txt
* temp.dat
This method works for folders and their contents, but not for specific files or wildcard.
"\USERS\$USER\AppData\Local\test_dir_cmd\folder\folder1",
"\USERS\$USER\AppData\Local\test_dir_cmd\folder\*.txt",
"\USERS\$USER\AppData\Local\test_dir_cmd\*\file.db",
"\USERS\$USER\AppData\Local\test_dir_cmd\temp.dat”
This is an example of how the list of files I'll need to get is presented and I'll need to work with.
Errors given:
Copy-Item : Illegal characters in path.
At F:\P2P.ps1:37 char:1
+ Copy-Item "C:$path" -Destination "$triage_location\$path" -Force -Rec ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [Copy-Item], ArgumentException
+ FullyQualifiedErrorId : System.ArgumentException,Microsoft.PowerShell.Commands.CopyItem
Command
Full script used for context:
$triage_location = "C:\temp\output\Triage\c"
ForEach-Object { #Looping through C:\Users to find folders that begin with numbers only and add to an array called $users
$users = #(Get-ChildItem -Path 'C:\Users'| Where-Object { $_.Name -match '^c+' } | Select -ExpandProperty Name)
}
Write-Host "users = $users"
write-host ""
$path_array = foreach ($user in $users) { # Loop through contents of users array and add each user to known locations
#(
"\USERS\$USER\AppData\Local\test_dir_cmd\folder\folder1",
"\USERS\$USER\AppData\Local\test_dir_cmd\folder\*.txt",
"\USERS\$USER\AppData\Local\test_dir_cmd\*\file.db",
"\USERS\$USER\AppData\Local\test_dir_cmd\temp.dat”
)
}
Write-Host "path_array = $path_array"
write-host ""
foreach ($path in $path_array) {
$a = Test-Path -Path "C:$path" # Creating variable called 'a' and setting it to Test-path value which is either True/False
if ($a -eq "True") # Test if browser location paths exist or not. If a returns True/False...
{
Write-Host "C:$path exists"
if(!(Test-Path -Path "$triage_location"))
{
New-Item -ItemType Directory -Path $triage_location
}
Copy-Item "C:$path" -Destination "$triage_location\$path" -Force -Recurse
}
else
{Write-Host "C:$path doesn't exist"}
}
if(Test-Path -Path "C:\temp\output\Triage")
{
Write-Host ""
Write-Host "Creating relevant .ZIP"
Compress-Archive -Path 'C:\temp\output\Triage' -DestinationPath 'C:\temp\output\P2P.zip' -Force # put zip in documents
}
Any help and advice on how I can fix this would be greatly appreciated!

The issue is that you are not joining the paths well. You do this:
-Destination "$triage_location\$path"
At that point $triage_location is C:\temp\output\Triage\c and $path is something like \USERS\TMTech\AppData\Local\test_dir_cmd\folder\folder1. You just make the path with string expansion but since $path starts with a \ and you include that in your string, so your string comes out looking like this:
"C:\temp\output\Triage\c\\USERS\TMTech\AppData\Local\test_dir_cmd\folder\folder1"
Use Join-Path instead:
Copy-Item (Join-Path 'C:\' $path) -Destination (Join-Path $triage_location $path) -Force -Recurse

Related

Move-Item causes "File Already Exists" error, despite folder having been deleted

I have some code which deletes a folder, then copies files from a temporary directory to where that folder had been.
Remove-Item -Path '.\index.html' -Force
Remove-Item -Path '.\generated' -Force -Recurse #folder containing generated files
#Start-Sleep -Seconds 10 #uncommenting this line fixes the issue
#$tempDir contains index.html and a sub folder, "generated", which contains additional files.
#i.e. we're replacing the content we just deleted with new versions.
Get-ChildItem -Path $tempDir | %{
Move-Item -Path $_.FullName -Destination $RelativePath -Force
}
I get an intermittent error, Move-Item : Cannot create a file when that file already exists. on the Move-Item line for the generated path.
I've been able to prevent this by adding a hacky Start-Sleep -Seconds 10 after the second Remove-Item statement; though that's not a great solution.
I assume the issue is that the Remove-Item statement completes / code moves on to the next line, before the OS has caught up with the actual file deletion; though that seems odd/worrying. NB: There are ~2,500 files in the generated folder (all between 1-100 KBs).
There are no other processes accessing the folders (i.e. I've even closed my explorer windows & tested with this directory being excluded from my AV).
I've considered other options:
using Copy-Item instead of Move-Item. I don't like this as it requires creating new files when they're not required (i.e. a copy is slower than a move)... It's faster than my current sleep hack; but still not ideal.
deleting the files & not the folder, then iterating through the subfolders & copying files to the new locations. This would work, but is a lot more code for something that should be simple; so I don't want to pursue that option.
Robocopy would do the trick; but I'd prefer a pure PowerShell solution. This is the option I'll eventually pick if there is no clean solution though.
Question
Has anyone seen this before?
Is it a bug, or have I missed something?
Is anyone aware of a fix / good workaround?
Update
Running the remove in a separate job (i.e. using the code below) did not resolve the issue.
Start-Job -ScriptBlock {
Remove-Item -Path '.\index.html' -Force
Remove-Item -Path '.\generated' -Force -Recurse #folder containing generated files
} | Wait-Job | Out-Null
#$tempDir contains index.html and a sub folder, "generated", which contains additional files.
#i.e. we're replacing the content we just deleted with new versions.
Get-ChildItem -Path $tempDir | %{
Move-Item -Path $_.FullName -Destination $RelativePath -Force
}
Update #2
Adding this works; i.e. rather than waiting a fixed time, we wait for the path to be removed / checking every second. If it's not removed after 30 seconds we assume it's not going to be; so carry on regardless (which will cause the move-item to throw an error which gets handled elsewhere).
# ... remove-item code ...
Start-Job -ScriptBlock {
param($Path)
while(Test-Path $Path){start-sleep -Seconds 1}
} -ArgumentList '.\generated' | Wait-Job -Timeout 30 | Out-Null
# ... move-item code ...
In the end I settled for this solution; not perfect, but it works.
Remove-Item -Path '.\index.html' -Force
Remove-Item -Path '.\generated' -Force -Recurse #folder containing generated files
#wait until the .\generated directory is full removed; or until ~30 seconds has elapsed
1..30 | %{
if (-not (Test-Path -Path '.\generated' -PathType Container)) {break;}
Start-Sleep -Seconds 1
}
Get-ChildItem -Path $tempDir | %{
Move-Item -Path $_.FullName -Destination $RelativePath -Force
}
This does the same as the job in update #2 of the question; only doesn't require the overhead of a job; just loops until the file's removed.
Here's the above logic wrapped as a reuable cmdlet:
function Wait-Item {
[CmdletBinding()]
param (
[Parameter(Mandatory, ValueFromPipeline, HelpMessage = 'The path of the item you wish to wait for')]
[string]$Path
,
[Parameter(HelpMessage = 'How many seconds to wait for the item before giving up')]
[ValidateRange(1,[int]::MaxValue)]
[int]$TimeoutSeconds = 30
,
[Parameter(HelpMessage = 'By default the function waits for an item to appear. Adding this switch causes us to wait for the item to be removed.')]
[switch]$Remove
)
process {
[bool]$timedOut = $true
1..$TimeoutSeconds | %{
if ((Test-Path -Path $Path) -ne ($Remove.IsPresent)){$timedOut=$false; return;}
Start-Sleep -Seconds 1
}
if($timedOut) {
Write-Error "Wait-Item timed out after $TimeoutSeconds waiting for item '$Path'"
}
}
}
#example usage:
Wait-Item -Path '.\generated' -TimeoutSeconds 30 -Remove

Variables not getting assigned in Powershell

I have a piece of code to search for all .txt and .csv files in the a bunch of submit folders inside a main folder.
#Folder to check for files
$path="\\Myfolder-DEV\RI*"
#variable which contains the number of files to be processed
$NumOfFiles = 0
Get-ChildItem $path -Recurse | % {
if (($_.Attributes -eq "Directory") -and ($_.FullName -match "submit")) {
$myPath = $_.FullName -replace "\\", "/"
Write-Host $myPath;
$fileEntries = Get-ChildItem -Path $_.FullName -include *.txt, *.csv -recurse;
foreach($fileName in $fileEntries){
$myDfile = $fileName.FullName -replace "\\", "/"
Write-Host $myDfile;
$myOfile = $myDfile -replace '\.[^.\\/]+$'
Write-Host $myOfile;
$NumOfFiles = $NumOfFiles + 1
}
}
}
echo "`n$NumOfFiles files were processed`n"
When I run this code in DEV, it works fine but when I do it in QA by modifying the path like this, it is not going through the folders at all.
$path="\\Myfolder-QA\RI*"
It even fails to write a value to the variable here
Write-Host $myPath;
I've tried modifying the scope - Clearing the variables at the end of the previous script. Restarting ISE but nothing seems to work.
Edit:
When I change the main directory to $path="\\Myfolder-QA\", it seems to work but it is searching through folders that I don't want it to. I need to search through only those folders which start with a RI and it doesnt seem to work. Any idea why it could happen?
Any help is appreciated
Thanks,
Sree

Comparing filenames in an array to files in a file structure

$folder = filestructure
# Get a recursive list of all folders beneath the folder supplied by the operator
$AllFolders = Get-ChildItem -Recurse -Path $Folder |? {$_.psIsContainer -eq $True}
# Get a list of all files that exist directly at the root of the folder
# supplied by the operator
$FilesInRoot = Get-ChildItem -Path $Folder | ? {$_.psIsContainer -eq $False}
Foreach ($File in ($FilesInRoot))
{
#Notify the operator that the file is being uploaded to a specific location
if($Global:successfullymigrated -contains $File){
Write-Host $File
}
}
###this part doesn't work
foreach($CurrentFolder in $AllFolders)
{
# Set the FolderRelativePath by removing the path of the folder supplied
# by the operator from the fullname of the folder
$FolderRelativePath = ($CurrentFolder.FullName).Substring($Folder.Length)
$FileSource = $Folder + $FolderRelativePath
$FilesInFolder = Get-ChildItem -Path $FileSource | ? {$_.psIsContainer -eq $False}
# For each file in the source folder being evaluated, call the UploadFile
# function to upload the file to the appropriate location
Foreach ($File in ($FilesInFolder))
{
Write-Host $File
if($Global:successfullymigrated -contains $File){
Write-Host $File
}
}
}
My code above is supposed to go through a file structure and checks to see if any of the file names are in the array (which is an array of strings with file names in them). My code works for the root files, prints out all the files that are in the array but when we get to checking the files in the other folders beyond the root it doesn't work. Even though it outputs the files that are in the file structure. I am completely stuck.
Forgive me if I have misunderstood but I read this
My code above is supposed to go through a file structure and checks to see if any of the file names are in the array
And interpreted that as you are just looking for file paths for files that match exactly a list of names you provide.
So I have this sample which should do just that.
$Global:successfullymigrated = #("template.txt","winmail.dat")
$folder = "C:\temp"
Get-ChildItem $folder -recurse | Where-Object{$Global:successfullymigrated -contains $_.Name -and !$_.psIsContainer} | Select-Object -ExpandProperty FullName
You should be able to incorporate this into your own code. It outputs the full paths to the matching files. The example I have outputs file from root and substructure.
C:\temp\winmail.dat
C:\temp\docs\template.txt
!$_.psIsContainer is to ensure that we do not get folders returned in our results. If you have PowerShell 3.0 or above then that can be replaced by the -File switch of Get-ChildItem

Unable to delete file in script

Going out of my mind over here.
I have a script where I'm parsing a folder full of tifs, and breaking the files up into sub-folders to limit the number of pages to around 60 per folder. If a document is very large it gets its own folder.
The problem is that the process is locking up the files, so they cannot be deleted. Not every file though, most of them work fine, and my end clean-up portion of the script gets rid of everything else.
I wrote a lot of work-around sections to my code to fix this issue, and now it is pretty bad looking
#Large Documents
Get-ChildItem -Path "$directory" -recurse -filter "*.tif" | foreach {
$file = [System.Drawing.Bitmap]::FromFile($_.Fullname);
$pagecount = $file.GetFrameCount($file.FrameDimensionsList[0]);
if ($pagecount -gt $MaxSize){
$total = $total + $pagecount;
$name = $_.Basename;
New-Item $name -ItemType directory;
Copy-Item $_.fullname -Destination $name;
#Copy-Item $name".DS" -Destination $processingDir;
Write-Host "Sleeping in large doc loop";
$file.Dispose;
Write-Host "Dispose file object";
Write-Host $_.Fullname
$storename = $_.Fullname
$largeFiles = $largeFiles + $storename
Write-Host "Storing to array: " $largeFiles[$index];
$index = $index + 1;
sleep(15);
}
}
while ($delInd -lt $largeFiles.Count){
Write-Host "Deleting: " $largeFiles[$delInd];
Remove-Item $largeFiles[$delInd] -Force;
$delInd = $delInd + 1;
}
I'm absolutely perplexed by this. Any help is greatly appreciated.
As far as I understand with $file.Dispose you do not force underlying object to close the file. Dispose is a method and, in PowerShell (like in C#), to invoke a method you have to use (). So try $file.Dispose().
piece of advice : you can avoid ; at the end of the lines

move-item doesn't work in loop

ls *.gif | Foreach { $newname = $_.Name -replace '\[','' -replace '\]',''
write-host $_.Name $newname
move-Item -Path $_.Name -Destination $newname; }
ls *.gif
So while trying to help someone rename files with [], I found out move-item doesn't work in a loop. It seems to work just fine outside the loop.
Ideas?
Update: Based on the comment below, I want to clarify this: The special characters in the file names require you to use -LiteralPath parameter. -Path cannot handle those characters. Outside a loop, -Path works since you are escapting the special characters using `. This isn't possible when walking through a collection.
In a loop, you need to use -LiteralPath parameter instead of -Path.
-LiteralPath <string[]>
Specifies the path to the current location of the items. Unlike Path, the value of
LiteralPath is used exactly as it is typed. **No characters are interpreted as
wildcards. If the path includes escape characters, enclose it in single quotation
marks.** Single quotation marks tell Windows PowerShell not to interpret any
characters as escape sequences.
SO, this will be:
GCI -Recurse *.txt | % { Move-Item -LiteralPath $_.FullName -Destination "SomenewName" }
If you use the pipeline binding feature of PowerShell, you can make this much simpler and eliminate the need for the explicit Foreach-Object e.g.:
ls *.gif | Move-Item -Destination {$_ -replace '\[|\]',''} -WhatIf
This works because the LiteralPath parameter is set up to bind ByPropertyName. However you may wonder, where does it get a property by the name of "LiteralPath" from on the output of Get-ChildItem (alias ls). Well it doesn't find that property name, however the LiteralPath parameter has an alias of PSPath defined which does exist on each object output by Get-ChildItem. That's how it binds to the LiteralPath paramter. The other speed tip here is that because the Destination parameter is also pipeline bound (ByPropertyName), you can use a scriptblock to provide the value. And inside that scriptblock you have access to the pipeline object.
Inside the scriptblock, this uses the -replace operator to come up with the new name based on the original full name. While I could have used $_.FullName or even $_.Name in this case (assuming you want to essentially rename the files within the same dir), I use just $_. Since -replace is a string operator, it will coerce $_ to a string before using it. You can see what this would be by executing:
ls *.gif | Foreach {"$_"}
Which is the full path in this case but you have to be careful because you don't always get the full path e.g.:
ls | Foreach {"$_"}
displays just the filename. In your examples (rename to same dir) this doesn't matter but in other cases it does. It is probably a good practice just to be explicit and use $_.Name or $_.FullName in a script but when hacking this stuff out at the console, I tend to use just $_. The saying: it's a sharp stick, don't poke your eye out applies here. :-)
You can find the "official" informations about the role of "[" in Path strings on this Microsoft article.
Or look in google for : Windows PowerShell Tip of the Week : "Taking Things (Like File Paths) Literally".
The only tip wich was not clear for me is that Rename-Item does not support LiteralPath, and that we can use Move-Item to rename files or directories.
JP
This worked for me (atleast in my situation.
Move-Item -literalpath $_.FullName -Destination ( ( ( (Join-Path -Path (Get-Location) -ChildPath $_.BaseName) -replace "\[","`[") -replace "\]","`]") )
Had hundreds of movies and it associated subtitles stored without folders. Decided to put each of the movies and subtitles in their own folders
Full Code
Get-ChildItem -File | %
{
if(Test-Path -LiteralPath ( Join-Path -Path (Get-Location) -ChildPath $_.BaseName ))
{
Move-Item -literalpath $_.FullName -Destination ( ( ( (Join-Path -Path (Get-Location) -ChildPath $_.BaseName) -replace "\[","`[") -replace "\]","`]") )
}
else
{
New-Item ( Join-Path -Path (Get-Location) -ChildPath $_.BaseName ) -ItemType Directory
Move-Item $_ -Destination ( Join-Path -Path (Get-Location) -ChildPath $_.BaseName )
}
}

Resources