I have to find matches over 3 CSVs. It is to find out whether users have AccessRights on PublicFolders in Exchange 2016. For ease of use I have already searched and stored all the needed values in 3 CSVs
"PF-Folder_Full.csv": a list of all the Publich Folders (more than 5000)
"PF-Mailboxes.csv": a list of all the users (around 50)
"PF-Permissions.csv": the result of
Get-PublicFolderClientPermission -Identity $Folder.Identity
looped through all the Public Folders (that takes ages)
I have written a script that does the job but even on a fast computer it is extremely slow because it has too loop through all the Public Folders and through all the users and then find a match for both values in the permissions
$Folders = Import-Csv -Path ".\PF-Folder_Full.csv" -Encoding Unicode
$Mailboxes = Import-Csv -Path ".\PF-Mailboxes.csv" -Encoding Unicode
$Permissions = Import-Csv -Path ".\PF-Permissions.csv" -Encoding Unicode
foreach ($Folder in $Folders) {
foreach ($Mailbox in $Mailboxes) {
$Permission = $Permissions | where {
($_.Identity -eq $Folder.Identity) -and
($_.User -eq $Mailbox.DisplayName)
}
if ($Permission) {
# some code
} else {
# some other code
}
Remove-Variable Permission
}
}
Is there a way to speed-up things? Possibly through the use of regular expressions.
I couldn't find any example that allows for extended matches between multiple arrays.
As Kory Gill mentioned in the comments: build two hashtables from the Identity and DisplayName properties from the first two CSVs:
$Folders = #{}
Import-Csv -Path ".\PF-Folder_Full.csv" -Encoding Unicode | ForEach-Object {
$Folders[$_.Identity] = $_
}
$Mailboxes = #{}
Import-Csv -Path ".\PF-Mailboxes.csv" -Encoding Unicode | ForEach-Object {
$Mailboxes[$_.DisplayName] = $_
}
Then process the third CSV using these hashtables for lookups:
$Permissions = Import-Csv -Path ".\PF-Permissions.csv" -Encoding Unicode
foreach ($p in $Permissions) {
if ($Folders.Contains($p.Identity) -and $Mailboxes.Contains($p.User)) {
# some code
} else {
# some other code
}
}
If you want code run just once per unique identity/user combination you could build a hashtable with the combined identity and mailbox name for filtering:
$Folders = Import-Csv -Path ".\PF-Folder_Full.csv" -Encoding Unicode |
Select-Object -Expand Identity
$Mailboxes = Import-Csv -Path ".\PF-Mailboxes.csv" -Encoding Unicode |
Select-Object -Expand DisplayName
$fltr = #{}
foreach ($f in $Folders) {
foreach ($m in $Mailboxes) {
$fltr["$f $m"] = $true
}
}
and then group the records from the third CSV:
Import-Csv -Path ".\PF-Permissions.csv" -Encoding Unicode |
Group-Object Identity, User |
ForEach-Object {
if ($fltr.Contains($_.Name)) {
# some code
} else {
# some other code
}
}
I'm relatively new to powershell and coding and am having issues accessing the values in an array. I'm trying to loop thru a set of files using foreach and count the number of messages in each file. And then have the count for each file put in to an array so I can assign it to a variable. When I do write-host $data[0] it returns all the values. If I do write-host $data1 it returns nothing. It seems like these values are all being stored as one instead of as individual numbers. How do I get each value and then assign it to a variable. Any help would be appreciated.
$FilePath = 'some file path here'
$TodaysDate = (Get-Date -format "MM-dd-yyyy")
ForEach($file in Get-ChildItem $FilePath -exclude *.ps1,*.xml,*.xls | Where-Object {$_.LastWriteTime -ge $TodaysDate})
{
$data = ,#(Get-Content $file | Where-Object {$_.Contains("MSH|")}).Count
write-host $data[0]
}
exit
powershell result
In this line:
$data = ,#(Get-Content $file | Where-Object {$_.Contains("MSH|")}).Count
you are creating an array of a single element (the count). What you want to do is add to $data each time:
$data += ,#(Get-Content $file | Where-Object {$_.Contains("MSH|")}).Count
But given your description, I think you may want a hashtable, using the filename as a key:
$data = #{} #init hashtable
ForEach($file in Get-ChildItem $FilePath -exclude *.ps1,*.xml,*.xls | Where-Object {$_.LastWriteTime -ge $TodaysDate})
{
$data[$file] = #(Get-Content $file | Where-Object {$_.Contains("MSH|")}).Count
}
write-output $data
I am trying to build an array with the results of the following powershell script. I would then like to only output the unique results in that array (I am expecting duplicates). I tried but it does not work. One thing I noticed is that my array is being converted to a character array. So if I call a single object in the array ($array[1]) it only displays one character, which may be why the select -unique is not working.
Please help.
$servers="Server1","server2"
$regex="(\.\d\d\d.*\()"
$path = "SomePath"
$yesterday = (get-date).Date.AddDays(-1)
Foreach ($server in $servers) {Get-ChildItem -Path $path -recurse | where { $_.LastWriteTime -le $yesterday } | ForEach-Object { Get-Content -Path "$_" -TotalCount 2 | select-string $regex | % { $_.Matches } | % { $array += $_.Value.Substring(5,6) } }}
$array|select -Unique
If you initialize the $array value as an array, then your code will work. Without this knowledge, PowerShell is treating $array += like concatenating strings together.
$array = #()
$servers="Server1","server2"
$regex="(\.\d\d\d.*\()"
$path = "SomePath"
$yesterday = (get-date).Date.AddDays(-1)
Foreach ($server in $servers) {Get-ChildItem -Path $path -recurse | where { $_.LastWriteTime -le $yesterday } | ForEach-Object { Get-Content -Path "$_" -TotalCount 2 | select-string $regex | % { $_.Matches } | % { $array += $_.Value.Substring(5,6) } }}
$array|select -Unique
We have a backup that runs every other day, but the files are large and we want to just remove every other one once we get a certain amount of backup files with our file signature.
I've tried this:
$Drive = "E:\temp\"
$deleteTime = -42;
$limit = (Get-Date).AddDays($deleteTime)
#this is finding the correct files but I don't think it's really in an array
$temp1 = Get-ChildItem -Path $Drive -filter "*junk.vhd*" | Where-Object {$_.LastWriteTime -lt $limit} | Select -Expand Name
for($i=$temp1.GetLowerBound(0); $i -le $temp1.GetUpperBound(0); $i+=2) {
Write-Host "removing $temp1[$i]" #this is listing the entire array with a [0] for first one and the third [2] element also, whether I cast to an array or not
}
I tried this instead of the above (Get-ChildItem) line currently but it listed the entire set of junk files for [0] instead of just the first junk.vhd at [0]:
[array]$temp1 =#( Get-ChildItem -Path $Drive -filter "*junk.vhd*" | Where-Object {$_.LastWriteTime -lt $limit} | Foreach-Object {$_.Name} )
I tried this too:
$limit = (Get-Date).AddDays(-42)
$list = (dir -Filter *junk.ps1 | where LastWriteTime -lt $limit).FullName
$count = $list.Length
for ($i = 0; $i -lt $count; $i += 2)
{
Write-Verbose "[$i] $($list[$i])"
#it's not getting in here because I'm not sure how
#to add the $Drive location and list is empty
}
Does anyone have a suggestion how to get an array of the filenames from $Drive location with the signature *junk.vhd so I can loop through them and remove every other one?
An internet search isn't turning much up.
This works for me:
$deleteTime = -12;
$limit = (Get-Date).AddDays($deleteTime)
$t = Get-ChildItem -Path $pwd -filter "p*.txt" | Where-Object {$_.LastWriteTime -lt $limit} | Select -Expand Name
foreach ($a in $t) { Write-Host "Name : $a" }
What have I missed from what you were looking for?
(Obviously, you will need to maintain a counter and do some modulo arithmetic in the body of the foreach() statement... )
This works, too:
for($i=$t.GetLowerBound(0); $i -le $t.GetUpperBound(0); $i+=2) {
$n = $t[$i]
Write-Host "removing $n"
}
Input:
1116559 P1303251287 20130325225906CD 13013822 1 0000
1104220 P1303250282 20130325070119CD 1 0000
1064743 P1303251094 20130325191600CD 0 0000
1100819 P1303250369 20130325091722CD 0 0000
1101405 P1303250051 20130325010740CD 2 0000
What I get from my attempt:
$lines = Get-Content "filenamehere.txt"
ForEach ($x in $lines) {
$y = "$($x[0..9] -join '')|$($x[10..23] -join '')|$($x[24..42] -join '')|
$($x[43..53] - join '')|$($x[54..57] -join '')|$($x[58..61] -join '')|
$($x[126..138] -join '')"
$z = $y -join '|'
Write-Output $z | Out-File -FilePath "foo.txt" -Append}
I get:
1116559 |P1303251287 |20130325225906CD |13013822 |1 |0000|
1104220 |P1303250282 |20130325070119CD | |1 |0000|
1064743 |P1303251094 |20130325191600CD | |0 |0000|
1100819 |P1303250369 |20130325091722CD | |0 |0000|
1101405 |P1303250051 |20130325010740CD | |2 |0000|
I don't mind the trailing spaces as long as I can get into this format. But "Get-Content" parse my data into an array and importing to SQL gives me an error. Question is, how can I convert this into CSV?
Output should be:
1116559|P1303251287|20130325225906CD|13013822|1|0000
1104220|P1303250282|20130325070119CD| |1|0000
1064743|P1303251094|20130325191600CD| |0|0000
1100819|P1303250369|20130325091722CD| |0|0000
1101405|P1303250051|20130325010740CD| |2|0000
I'd do that with -replace
$Regex = '(.{7})\s{3}(.{11})\s{3}(.{16})\s{3}(.{8})\s{3}(.{1})\s{3}(.{4})'
$Replace = '$1|$2|$3|$4|$5|$6'
(Get-Content "filenamehere.txt") -replace $Regex,$Replace |
Set-Content "foo.txt"
Using your sample you can use trim()
$lines = Get-Content "c:\temp\filenamehere.txt"
ForEach ($x in $lines)
{
$y = "$(($($x[0..9] -join '')).trim())|$(($($x[10..23] -join '')).trim())|$(($($x[24..42] -join '')).trim())|$(($($x[43..53] -join '')).trim())|$(($($x[54..57] -join '')).trim())|$(($($x[58..61] -join '')).trim())|$(($($x[126..138] -join '')).trim())"
$z = $y -join '|'
Write-Output $z | Out-File -FilePath "c:\temp\foo.txt" -Append
}
Perhaps it remove too much spaces.
I output
1116559|P1303251287|20130325225906CD|13013822|1|0000|
1104220|P1303250282|20130325070119CD||1|0000|
1064743|P1303251094|20130325191600CD||0|0000|
1100819|P1303250369|20130325091722CD||0|0000|
1101405|P1303250051|20130325010740CD||2|0000|
Which should be better on a CSV point of view.
#echo off
for /F "tokens=1-6" %%a in (input.txt) do (
if "%%f" neq "" (
echo %%a^|%%b^|%%c^|%%d^|%%e^|%%f
) else (
echo %%a^|%%b^|%%c^| ^|%%d^|%%e
)
)
Output:
C:\> test.bat
1116559|P1303251287|20130325225906CD|13013822|1|0000
1104220|P1303250282|20130325070119CD| |1|0000
1064743|P1303251094|20130325191600CD| |0|0000
1100819|P1303250369|20130325091722CD| |0|0000
1101405|P1303250051|20130325010740CD| |2|0000
Using a ConvertFrom-FixedLengths function you could just do:
Get-Content "C:\input.txt" |
ConvertFrom-FixedLengths 10,14,19,11,4,4 -Trim |
Foreach { #($_.Column1, $_.Column2, $_.Column3, $_.Column4.PadLeft(8, ' '), $_.Column5, $_.Column6) -Join "|" } |
Out-File -FilePath "c:\output.txt"
Or, of course, if you want to create a csv-file with the | character as delimiter you could just do:
Get-Content "C:\input.txt" |
ConvertFrom-FixedLengths 10,14,19,11,4,4 -Trim |
Select Column1, Column2, Column3, #{ N = "Column4"; E = { $_.Column4.PadLeft(8) } }, Column5, Column6 |
Export-Csv -Path "C:\Output.csv" -NoTypeInformation -Delimiter "|"
Or, to make it even simpler, if you want a csv-file and don't need to pad the fourth column with spaces, you could just skip the `Select´ line in the last sample, making it into:
Get-Content "C:\input.txt" |
ConvertFrom-FixedLengths 10,14,19,11,4,4 -Trim |
Export-Csv -Path "C:\Output.csv" -NoTypeInformation -Delimiter "|"
For a decent solution, you need to handle the content as fixed length fields, the other answers here do that.
If you know only column 4 might be blank, you can bodge it for a one-off script by replacing an 11 character space with a comma (which will do nothing on rows where column 4 has content), then replacing spaces with commas:
Get-Content "data.txt" | % { ($_ -replace "\s{11}", ",") -replace "\s+", "," } > out.txt
Sample output:
1116559,P1303251287,20130325225906CD,13013822,1,0000
1104220,P1303250282,20130325070119CD,,1,0000
1064743,P1303251094,20130325191600CD,,0,0000
1100819,P1303250369,20130325091722CD,,0,0000
1101405,P1303250051,20130325010740CD,,2,0000
Working code..
CD 'C:\\FOLDERPATH\'
$filter = "FILE_NAME_*.txt"
$columns = 11,22,32,42,54
## DO NOT NEED TO REVERSE [array]::Reverse($columns) #too lazy to re-write array after finding out I need to iterate in reverse
$files = get-childitem ./ |where-object {$_.Name -like $filter}
$newDelimiter = '|'
foreach($file in $files)
{
$file
$csvFile = 'C:\\FOLDERPATH\NEW_' + $file.BaseName + '.txt'
if (!(get-childitem ./ |where-object {$_.Name -like $csvFile})) #check whether file has been processed
{
$content | ForEach {
$line = $_
$counter = 0
$columns | ForEach {
$line = $line.Insert($_+$counter, $newDelimiter)
$counter = $counter + 1
}
$line = $line.Trim($newDelimiter)
$line
} | set-content $csvFile
}
}