I am fairly new to PowerShell. I have created an exe that can be ran by a coworker that will go to 8 separate sql servers, to individual directories and check them for a list of created files. My current code checks for age less than one day and that it's not empty, however, I have been presented with a new problem. I need to be able to take a list in a text file/array/csv/etc and compare the list to the directory to ensure that there are not any additional files in the folder. All of the information is then formatted, exported to a file, and then emailed to certain recipients.
My question is, how do I create a script that works using this idea; preferably without considerable change to my current code as it is fairly length considering the sub directories of the 8 sql servers.. (current script redacted is as follows):
$today = get-date
$yesterday = $today.AddDays(-1)
$file_array = "XXX_backup*.trn", "XXX_backup*.bak", "XXY_backup*.trn", "XXY_backup*.bak"
$server_loc = \\hostname\e$\
$server_files = #(get-childitem $server_loc | where-object {$_.fullname -notmatch "(subdirectory)})
$server_complete_array = #("Files Directory ($server_loc)", " ")
foreach ($file in $files) {
IF ($file.length -eq 0) {
[string]$string = $file
$server_complete_Array = $server_complete_Array + "$string is empty."
}
elseif ($file.LastWriteTime -lt $yesterday) {
[string]$string = $file
$server_complete_Array = $server_complete_Array + "$string is old."
}
else {
[string]$string = $file
$server_complete_Array = $server_complete_Array + "$string is okay."}
}
Thank you for your help in advance. :)
Bit of a hack, but it should work with what you already have:
$file_array = "XXX_backup*.trn", "XXX_backup*.bak", "XXY_backup*.trn", "XXY_backup*.bak"
$FileNames = $server_files | select -ExpandProperty Name
$FileCheck = ('$Filenames ' + ($file_array | foreach {"-notlike '$_'"}) -join ' ')
Since you're already using wildcards in your file specs, this works with that by creating a command which filters the file name array through a series of -notlike operators, one for each file spec.
$FileCheck
$Filenames -notlike 'XXX_backup*.trn' -notlike 'XXX_backup*.bak' -notlike 'XXY_backup*.trn' -notlike 'XXY_backup*.bak'
Each one will filter out the names that match the file spec and pass the rest onto the next -notlike operator. Whatever falls out the end didn't match any of them
At this point $FileCheck is just a string, and you'll need to use Invoke-Expression to run it.
$ExtraFiles = Invoke-Expression $FileCheck
and then $ExtraFiles should contain the names of all the files that did not match any of the file specs in $file_array. Since the filters are being created from $file_array, there's no other maintenance to do if you add or remove filespecs from the array.
Related
I have a script where I get all of the folders in d:\folder\*\*\ where the name is -like "*\Log". I then split the folder paths apart to run through wmi to get the corresponding services. After that I'm wanting to split apart the PathName property from $Services so I get everything before the \xxxxx.exe and add \log to the end of the result. Eventually I'll then use those paths to do some compression and archiving of files via a gci.
For whatever reason when I run the script below I the previous loops $LocalLogVar without "log" appended and the current loops LocalLogVar with log appended. I'm sure I'm doing something wrong that's blatantly obvious to somebody out there. If somebody could point me in the right direction on this it'd be much appreciated! I also apologize for the word vomit here, I've been looking at this script all day and my brain's pretty much used up.
A couple of notes:
The number of words in the paths vary which is why I can't manually do $LocalLogVar = "$Var1\$Var2\$Var3\Log"
If I don't have the [array] in front of $LogFolders the object type becomes a string and I get the previous loop's $LocalLogVar without "log" appended combined with the current loop's $LocalLogVar
I tried doing [collections.arraylist]$LogFolders=#() with no success
c:\folder is a shortcut to d:\folder, which is why there's c:\folder\xxx and d:\folder\xxx in the list below
SplitCount is -1 because I don't want the .exe from the path, I just want the folder structure
The naming convention for the string before .exe varies so I can't use an enumerated counter.
Example of first bullet:
word7-word8 #This is the previous loop's $LocalLogVar w/o "log" appended
C:\folder\word5\word6\word9-word8\log #This is the current loop's $LocalLogVar w/ "log" appended.
Example of the second bullet:
word7-word8C:\folder\word5\word6\word9-word8\log
What I should be getting:
D:\folder\word-anotherword\word7-word8\log
D:\folder\word-anotherword\word9-word8\log
C:\folder\word1\word7-word8\log
C:\folder\word1\word9-word8\log
C:\folder\word2\word7-word8\log
C:\folder\word2\word9-word8\log
D:\folder\word2\word10-word11\log
D:\folder\word2\word12-word8\log
C:\folder\word3\word7-word8\log
C:\folder\word3\word9-word8\log
D:\folder\word4\word7-word8\log
C:\folder\word4\word9-word8\log
C:\folder\word5\word6\word7-word8\log
C:\folder\word5\word6\word9-word8\log
C:\folder\word5\word6\word7-word8\log
C:\folder\word5\word6\word9-word8\log
$Folders = Get-ChildItem D:\folder\*\*\ -Directory -Recurse -Verbose `
| Where-Object { $_.fullname -like "*\Log" }
$2 = #()
$LogFolders = #()
foreach ($folder in $folders) {
$ServName = $folder.fullname.split('\')[2]
$ServType = $folder.fullname.split('\')[3]
$ServNameCheck = "*$($ServName.replace('-',' '))*"
$ServTypeCheck = "*$($ServType.replace('-',' '))*"
$PathName = Get-WmiObject -ClassName Win32_Service `
| Where-Object { $_.caption -like "$ServNameCheck" -and $_.caption -like "$ServTypeCheck" } `
| Select-Object Name, Caption, #{n = 'PathName'; e = { ($_.PathName).trim('"') } }
$2 += $PathName
}
$Services = $2 | Sort-Object pathname | Get-Unique -AsString
foreach ($ServPath in $services.pathname) {
$LocalLogVar = #()
if (Get-Variable `
| Where-Object { $_.name -match "^Split([0-9]|10)$" }) {
Get-Variable `
| Where-Object { $_.name -match "^Split([0-9]|10)$" } | Remove-Variable -Force
}
[int]$SplitCount = $ServPath.split('\').count
[int]$SplitCountCheck = $SplitCount - 1
$x = 0
do {
New-Variable -Name "Split$x" -Value "$($ServPath.split('\')[$x])"
$RegEx = "Split$x"
$LogFolderName = Get-Variable | Where-Object { $_.name -match $RegEx } | Select-Object value
[string]$LogFolders = $LogFolderName.value.ToString()
$LocalLogVar += $LogFolders + '\'
$x++
} until ($x -eq $SplitCountCheck)
$LocalLogVar = $LocalLogVar
$LocalLogVar = $LocalLogVar + "log"
[array]$LogFolders += $LocalLogVar
}
Wow, so that's a script. Kind of hard to follow, since some of it seems needlessly complex. I'm not sure if it will accomplish what you're looking for, but that's because you were super vague with your folder descriptions. Do the folders always start like this:
D:\folder<Service Short Name><Service Long Name>...\logs
If not you could be in trouble. The last four items on your example list of what you expect to see don't look like they are like that. I think the way your folders are laid out are like this:
D:\folder...<Service Short Name><Service Long Name>\logs
The difference being where the extra folders are located. If they're before the service like I think they are your script will miss things.
Now, on to getting your list that you want. What I see from looking at your script is that you get a folder list for all folders under D:\folder\*\*\ named 'log'. Then you split out the 3rd and 4th folders to get a service's short name, and long name respectively. Then one by one you pull a list of all services from WMI, and filter for just the service that matches the name and caption (short name, and long name) referred to by the folders. After that you make sure you only have one listing of any given service.
Regarding this first part of the script, you can make it faster by letting the file system provider filter things for you. Instead of pulling a folder list of everything and then filtering for paths that end in '\log', you should use the -filter parameter of the Get-ChildItem cmdlet like this:
$Folders = Get-ChildItem C:\temp\*\*\ -Directory -Recurse -Verbose -Filter 'log'
Then you should query WMI one time, save the result, then pick and choose from there based on your folders. Something like:
[array]$2 = foreach ($folder in $folders) {
$ServName,$ServType = $folder.fullname.split('\')[2,3] -replace '-',' '
$PathName = $AllServices |
Where-Object { $_.caption -like "*$ServName*" -and $_.caption -like "*$ServType*" } |
Select-Object Name, Caption, #{n = 'PathName'; e = { $_.PathName -replace '^(\w\S+) .*','$1' -replace '^([''"])([^\1]+)\1.*','$2' } }
}
$Services = $2 | Sort-Object pathname | Get-Unique -AsString
I did a little regex magic to clean up the pathname instead of just .trim('"') since this gets rid of parameters in the service execution, and cleans paths that are enclosed in single quotes not just double quotes. If what you have works for you feel free to keep it, but this is a little more capable. It may be worth noting that Get-Unique is case sensitive, so 'C:\folder\word3\word9-word8' and 'C:\folder\word3\word9-Word8' are different. You might want to do a .ToUpper() on your paths before you look for unique ones.
Once you have your array of services you loop through them, splitting the file path, reassembling it, and finally adding 'log' to the end of it. That was your way to remove the executable from the path. There's a cmdlet that was designed to do just that: split-path. Use that with Join-Path and that whole last loop gets much simpler:
[array]$LogFolders = foreach ($ServPath in $services.pathname) {
Join-Path (Split-Path $ServPath) 'log'
}
Lastly, try not to use +=, since PowerShell has to rebuild the whole array each time you do that. You'll notice I moved the $Variable = bit outside the loop in places that you do that.
My array has a lot of properties but I'm only looking to edit one. The goal is to remove domains from the hostname but I haven't been able to get it working. This data is being returned from a REST API and there are thousands of assets that contain the same type of data (JSON content). The end goal is to compare assets pulled from the API and compare it to assets in a CSV file. The issue is that the domain may appear on one list and not the other so I'm trying to strip the domains off for comparison. I didn't want to iterate through both files and the comparison has to go from the CSV file to the API data hence the need to get rid of the domain altogether.
There are other properties in the array that I will need to pull from later. This is just an example of one of the arrays with a few properties:
$array = #{"description"="data"; "name"="host1.domain1.com"; "model"="data"; "ip"="10.0.0.1"; "make"="Microsoft"}
#{"description"="data"; "name"="host2.domain2.com"; "model"="data"; "ip"="10.0.0.2"; "make"="Different"}
#{"description"="data"; "name"="10.0.0.5"; "model"="data"; "ip"="10.0.0.5"; "make"="Different"}
The plan was to match the domain and then strip using the period.
$domainList = #(".domain1.com", ".domain2.com")
$domains = $domainList.ForEach{[Regex]::Escape($_)} -join '|'
$array = $array | ForEach-Object {
if($_.name -match $domains) {
$_.name = $_.name -replace $domains }
$_.name
}
You may do the following to output a new array of values without the domain names:
# starting array
$array = #("hosta.domain1.com", "hostb.domain2.com", "host3", "10.0.0.1")
# domains to remove
$domains = #('.domain1.com','.domain2.com')
# create regex expression with alternations: item1|item2|item3 etc.
# regex must escape literal . since dot has special meaning in regex
$regex = $domains.foreach{[regex]::Escape($_)} -join '|'
# replace domain strings and output everything else
$newArray = $array -replace $regex
Arrays use pointers so I needed to load the array and pipe through ForEach-Object and then set the object when logic was complete. Thanks for all of the help.
$domainList = #(".domain1.com", ".domain2.com")
$domains = $domainList.ForEach{[Regex]::Escape($_)} -join '|'
$array = $array | ForEach-Object {
if($_.name -match $domains) {
$_.name = $_.name -replace $domains }
$_.name
}
first time stackoverflower.
I have a need to remove 'ghost' entries from the PackageRegistrations of my software deployment GPOs.
What I mean by that is that there are more entries in the ADSI object than there are MSI/MST files associated to the GPO. i.e. what this blogger also seems to be experiencing http://justanotheritblog.co.uk/2016/11/15/list-msi-paths-from-software-installation-policies/ (I just found this when looking into my issue).
When nosing around the properites in ADSI, I stubmbled across 'msiScriptName', which seems to have a value of either 'A' or 'R'.
What I cannot seem to find, is any information as to what these values may represent.
Any ideas on what the 'A' and 'R' mean and/or how to correctily identify and/or remove the 'ghost' entries greatly recieved.
The reason for this is that I have a whole bunch of software deployment GPOs that need the file path updating, and rather than manually editing each one I wanted to use PowerShell to bulk update them - we are moving to DFS from fixed file server, so I need to update the msiFileList properties. This I can do, but do not want to waste processing overhead on irrelevant objects.
The following is rough code suggesting how I am doing this
$MSIFiles = #()
# Get all the SoftwareDeployment GPOs, indicated by a displayname continaing 'Install' and create an object for each MSI/MST associated to it.
$Packages = Get-GPO -All | Where-Object { $_.DisplayName -like "*Install*" } | Get-ADObjectGPOPackages -Domain 'skyriver.internal'
foreach ($p in $Packages)
{
$msiCount = ($p.msiFileList | Measure-Object).Count
$msiFileListNew = #()
for ($i = 0; $i -lt $msiCount; $i ++)
{
$msiFile = $p.msiFileList[$i] -replace 'hoth(01|01.skyriver.internal|02.skyriver.internal)','skyriver.internal\data'
$msiFileListNew += $msiFile
}
$Properties = [ordered]#{
'gpoDisplayName' = $p.gpoDisplayName
'PackageNumber' = $p.PackageNumber
'DisplayName' = $p.DisplayName
'CN' = $p.CN
'DistinguishedName' = $p.DistinguishedName
'Identity' = $p.Identity
'msiFileList' = $msiFileListNew
}
$obj = New-Object -TypeName psobject -Property $Properties
$MSIFiles += $obj
}
# Now make the replacements.
foreach ($m in $MSIFiles)
{
Set-ADObject -Identity $m.Identity -Server dagobah.skyriver.internal -Replace #{msiFileList = $m.msiFileList}
}
So far as I can tell, A is Advertised (ie available for install), and R is Remove. The Ghost packages probably have an R as they are no longer valid and are therefore to be uninstalled (I'm not sure if this only applies if the "uninstall when it falls out of scope" option is enabled before deleting?).
I need a batch file /script/tool to delete specified files in folder.
I have a folder with a lot of .xml files. It can contain files named difference of only a few characters (indicating the date).
aa_bb_000000001_2015_9_1.xml
aa_bb_000000001_2015_9_15.xml
aa_bb_000000001_2015_10_1.xml
aa_bb_000000002_2015_5_5.xml
aa_bb_000000002_2015_8_14.xml
aa_bb_000000002_2015_10_1.xml
aa_bb_000000005_2015_7_7.xml
.
.
The length of this part is 15 string
aa_bb_000000001
This part represents a date
2015_10_1
I need to delete all the files that part of the name with a date is earliest.
As a result batch should stay only files:
aa_bb_000000001_2015_10_1.xml
aa_bb_000000002_2015_10_1.xml
aa_bb_000000005_2015_7_7.xml
.
.
Here's one solution that's fairly short. To understand how the code works, it would be best to focus on what the Group-Object command does, what regular expressions are, and how they interact with the -match operator:
$Groups = Get-ChildItem "C:\XMLFiles\*.xml" | Group-Object {$_.Name.Substring(0, 15)}
$FilesToKeep = #{}
foreach ($Group in $Groups) {
$MaxDate = "00000000"
foreach ($FileInfo in $Group.Group) {
$FileInfo.name -match "(\d{4})_(\d{1,2})_(\d{1,2}).xml$" | Out-Null
$Date = $Matches[1]+([int]$Matches[2]).ToString("00")+([int]$Matches[3]).ToString("00")
if ($Date -gt $MaxDate) {
$MaxDate = $Date
$FilesToKeep[$Group.Name] = $FileInfo.FullName
}
}
}
Get-ChildItem "C:\XMLFiles\*.xml" | Where-Object {-not $FilesToKeep.ContainsValue($_.FullName)} | Remove-Item
Im interested in some ideas on how one would approach coding a search of a filesystem for files that match any entries contained in a master CSV file. I have a function to search the filesystem, but filtering against the CSV is proving harder than I expect. I have a csv with headers in it for Name & IPaddr:
#create CSV object
$csv = import-csv filename.csv
#create filter object containing only Name column
$filter = $csv | select-object Name
#Now run the search function
SearchSubfolders | where {$_.name -match $filter} #returns no results
I guess my question is this: Can I filter against an array within a pipeline like this???
You need a pair of loops:
#create CSV object
$csv = import-csv filename.csv
#Now run the search function
#loop through the folders
foreach ($folder in (SearchSubfolders)) {
#check that folder against each item in the csv filter list
#this sets up the loop
foreach ($Filter in $csv.Name) {
#and this does the checking and outputs anything that is matched
If ($folder.name -match $Filter) { "$filter" }
}
}
Usually CSVs are 2-dimensional data structures, so you can't use them directly for filtering. You can convert the 2-dimensional array into a 1-dimensional array, though:
$filter = Import-Csv 'C:\path\to\some.csv' | % {
$_.PSObject.Properties | % { $_.Value }
}
If the CSV has just a single column, the "mangling" can be simplified to this (replace Name with the actual column name):
$filter = Import-Csv 'C:\path\to\some.csv' | % { $_.Name }
or this:
$filter = Import-Csv 'C:\path\to\some.csv' | select -Expand Name
Of course, if the CSV has just a single column, it would've been better to make it a flat list right away, so it could've been imported like this:
$filter = Get-Content 'C:\path\to\some.txt'
Either way, with the $filter prepared, you can apply it to your input data like this:
SearchSubFolders | ? { $filter -contains $_.Name } # ARRAY -contains VALUE
The -match operator won't work, because it compares a value (left operand) against a regular expression (right operand).
See Get-Help about_Comparison_Operators for more information.
Another option is to create a regex from the filename collection and use that to filter for all the filenames at once:
$filenames = import-csv filename.csv |
foreach { $_.name }
[regex]$filename_regex = ‘(?i)^(‘ + (($filenames | foreach {[regex]::escape($_)}) –join “|”) + ‘)$’
$SearchSubfolders |
where { $_.name -match $filename_regex }
You can use Compare-Object to do this pretty easily if you are matching the actual Names of the files to names in the list. An example:
$filter = import-csv files.csv
ls | Compare-Object -ReferenceObject $filter -IncludeEqual -ExcludeDifferent -Property Name
This will print the files in the current directory that match the any Name in files.csv. You could also print only the different ones by dropping -IncludeEqual and -ExcludeDifferent flags. If you need full regex matching you will have to loop through each regex in the csv and see if it is a match.
Here's any alternate solution that uses regular expression filters. Note that we will create and cache the regex instances so we don't have to rely on the runtime's internal cache (which defaults to 15 items). First we have a useful helper function, Test-Any that will loop through an array of items and stop if any of them satisfies a criteria:
function Test-Any() {
param(
[Parameter(Mandatory=$True,ValueFromPipeline=$True)]
[object[]]$Items,
[Parameter(Mandatory=$True,Position=2)]
[ScriptBlock]$Predicate)
begin {
$any = $false
}
process {
foreach($item in $items) {
if ($predicate.Invoke($item)) {
$any = $true
break
}
}
}
end { $any }
}
With this, the implementation is relatively simple:
$filters = import-csv files.csv | foreach { [regex]$_.Name }
ls -recurse | where { $name = $_.Name; $filters | Test-Any { $_.IsMatch($name) } }
I ended up using a 'loop within a loop' construct to get this done after much trial and error:
#the SearchSubFolders function was amended to force results in a variable, SearchResults
$SearchResults2 = #()
foreach ($result in $SearchResults){
foreach ($line in $filter){
if ($result -match $line){
$SearchResults2 += $result
}
}
}
This works great after collapsing my CSV file down to a text-based array containing only the necessary column data from that CSV. Much thanks to Ansgar Wiechers for assisting me with that particular thing!!!
All of you presented viable solutions, some more complex than I cared for, nevertheless if I could mark multiple answers as correct, I would!! I chose the correct answer based on not only correctness but also simplicity.....