Array causing 'system.outofmemoryexception' - arrays

I am running the below script and it is causing exception of type 'system.outofmemoryexception' was thrown
I believe that it is due to the #Results array growing past the 2gb allocated to windows shell. Is there possibly a way to iterate through the results, or am I stuck with allocating more memory (which could ultimately be a lot...)?
$Path = "path to output"
### Get all PF
$publicFolders = Get-PublicFolder "Public Folder Name" -Recurse -resultsize unlimited | Select-Object *
### Array to contain results
$results = #()
###Begin looping through each PF and grab each user/group with AccessRights to that folder
$final = ForEach($pf in $publicFolders){
$perms = Get-PublicFolderClientPermission -Identity $pf.Identity | Where-Object {$_.User -notmatch "Default|Anonymous|S-X-X-XX"}
Foreach($perm in $perms){
$temp = [PSCustomObject]#{
MailFolderName = $pf.Identity
UserWithPerms = $perm.User
AccessRights = $perm | Select-Object -ExpandProperty AccessRights
}
$results += $temp
}
}
$final | Export-Csv $path -NoTypeInformation
Am I barking up the wrong tree?
Thanks in advance.

Use the ForEach-Object cmdlet instead of a foreach(){} loop statement for the outer loop - this way you can start piping the output to Export-Csv immediately instead of buffering it in an array:
$publicFolders |ForEach-Object {
$pf = $_
$perms = Get-PublicFolderClientPermission -Identity $pf.Identity | Where-Object {$_.User -notmatch "Default|Anonymous|S-X-X-XX"}
Foreach($perm in $perms){
[PSCustomObject]#{
MailFolderName = $pf.Identity
UserWithPerms = $perm.User
AccessRights = $perm | Select-Object -ExpandProperty AccessRights
}
}
} | Export-Csv $path -NoTypeInformation
Alternatively, flush the partial set of results to file after enumerating the permissions for each folder:
ForEach($pf in $publicFolders){
$perms = Get-PublicFolderClientPermission -Identity $pf.Identity | Where-Object {$_.User -notmatch "Default|Anonymous|S-X-X-XX"}
$results = Foreach($perm in $perms){
[PSCustomObject]#{
MailFolderName = $pf.Identity
UserWithPerms = $perm.User
AccessRights = $perm | Select-Object -ExpandProperty AccessRights
}
}
# flush the buffer before moving to the next set of permissions
$results |Export-Csv $path -NoTypeInformation -Append
}

Related

Powershell - Start-ThreadJob Ids Incrementing Very Quickly, Why?

New to Powershell and Stackoverflow. Here's my first Powershell Script that I'm trying to optimize to the best of my abilities. My goal is to have the code run as efficiently as possible. Any help/suggestions on that front would be much appreciated!
This script shows new 'Established' TCP Connections (Get-NetTCPConnection) and their associated DNS Hostnames (Resolve-DnsName). Each new Connection is compared to an array of previous Connections. If they have the same 'RemoteAddress', the DNS Hostname is copied over to the new Connection and displayed; otherwise it creates a new (Resolve-DnsName) (Start-ThreadedJob), and moves on to the next new Connection. Once a Job is 'Completed' it copies over the 'NameHost' and displays the Connection.
I have hit a roadblock in my understanding. When the code is running, the Job 'Ids' seem to be incrementing very quickly even though no new Jobs where created in between the last Job and the new Job.
To test the script, run it and visit any Site. Watch as the 'Id' increment very quickly. Please note that it will create a Log File in "C:\Temp\Active_Connections.csv"
$logFile = 'C:\Temp\Active_Connections.csv'
if (-not(Test-Path $logFile -PathType Leaf)){
New-Item -ItemType File -Force -Path $logFile | Out-Null
} else {
Clear-Content $logFile
}
$headersAdded = $true
$newConnections = #()
While ($true){
$connections = #(Get-NetTCPConnection)
foreach ($connection in $connections){
if ($connection.State -eq "Established"){
if ($newConnections.InstanceID -notcontains $connection.InstanceID){
if ($newConnections.RemoteAddress -notcontains $connection.RemoteAddress){
if ((Get-Job).Name -notcontains $connection.RemoteAddress){
Start-ThreadJob -Name $connection.RemoteAddress -ScriptBlock {param($remoteAddress) Resolve-DNSName -Name $remoteAddress} -ArgumentList $connection.RemoteAddress >$null
}else{
$job = Get-Job | Where-Object {$_.Name -eq $connection.RemoteAddress}
if ($job.State -eq "Completed"){
Add-Member -InputObject $connection -MemberType NoteProperty -Name "Id" -Value $job.Id -Force
Try {
$receivedJob = $job | Receive-Job -ErrorAction Stop
Add-Member -InputObject $connection -MemberType NoteProperty -Name "NameHost" -Value $receivedJob.NameHost -Force
}catch{
$na = "N/A"
Add-Member -InputObject $connection -MemberType NoteProperty -Name "NameHost" -Value $na -Force
}
#Remove-Job -Id $job.Id
}
}
}else{
foreach ($newConnection in $newConnections){
if ($newConnection.RemoteAddress -eq $connection.RemoteAddress){
Add-Member -InputObject $connection -MemberType NoteProperty -Name "NameHost" -Value $newConnection.NameHost -Force
}
}
}
}
if ($null -ne $connection.NameHost){
if ($headersAdded) {
$formatting = #{n='CreationTime';e={$_.CreationTime.ToString("h:mm:ss tt")}},'Id','LocalAddress','LocalPort','RemoteAddress','RemotePort','NameHost'
$properties = #{Expression="CreationTime";Width=13},#{Expression="Id";Width=4},#{Expression="LocalAddress";Width=15},#{Expression="LocalPort";Width=10;Alignment="Left"},#{Expression="RemoteAddress";Width=15},#{Expression="RemotePort";Width=10;Alignment="Left"},#{Expression="NameHost";Width=100}
($connection | Select-Object $formatting | Format-Table -Property $properties | Out-String).Trim() | Tee-Object -FilePath $logFile -Append
$headersAdded = $false
} else {
($connection | Select-Object $formatting | Format-Table -HideTableHeaders -Property $properties | Out-String).Trim() | Tee-Object -FilePath $logFile -Append
}
$newConnections += $connection
}
}
}
}
Please, let me know what I can do better and if you have any ideas as to why the Job Id's are incrementing so quickly between new Connections.
Appreciate the help,
Chris
I have no explanation for the jumps in job ID values. While it would be good to know the reason, pragmatically speaking, it isn't necessarily a problem.
Your code creates a tight loop which is best avoided.
The following is a PowerShell-idiomatic reformulation of your code that tries to get results as early as possible, while sleeping a fixed amount of time between tries (which you can obviously adjust).
The upshot is that the output objects won't necessarily be ordered chronologically.
The Id property (column) of the output objects reflects the original output order as returned by Get-NetTCPConnection
# NOTE: What is created is NOT a CSV file.
# It is a plain-text file in tabular format FOR THE HUMAN OBSERVER.
$logFile = 'C:\Temp\Active_Connections.csv'
& {
$newConnections = [ordered] #{} # (Ordered) hashtable that stores all new connections.
while ($true) {
# Look for new connections, and start a thread job for each
# in order to resolve the remote adddress to a domain name, if possible.
Get-NetTCPConnection |
Where-Object { $_.State -eq 'Established' -and -not $newConnections.Contains($_.InstanceID) } |
ForEach-Object {
$jb = Start-ThreadJob { Resolve-DNSName -Name ($using:_).RemoteAddress }
$newConnections[$_.InstanceID] =
$_ |
Select-Object CreationTime,
#{
n = 'Id'
e = { $jb.Id }
},
LocalAddress, LocalPort, RemoteAddress, RemotePort,
#{
n = 'NameHost'
e = { $jb }
}
}
# Sleep a little, to avoid a tight loop.
Start-Sleep -Milliseconds 300
# Look for thread jobs that have completed, and output
# the connection-info objects with the job result.
$newConnections.Keys |
ForEach-Object {
if (($obj = $newConnections[$_]) -and ($jb = $obj.NameHost).State -notin 'NotStarted', 'Running') {
# A completed job: get its result.
$result = try { $jb | Receive-Job -ErrorAction Stop } catch { #{ NameHost = 'n/a' } }
$jb | Remove-Job -Force # Remove the completed job.
$obj.NameHost = $result.NameHost # Update the object with the job result.
$obj # Output the updated object.
$newConnections[$_] = $null # No need to hang on to the object in the hasthable.
}
}
}
} |
Format-Table #{ Name = 'CreationTime'; Expression = { $_.CreationTime.ToString('h:mm:ss tt') }; Width = 13 },
#{Expression = "Id"; Width = 4 },
#{Expression = "LocalAddress"; Width = 15 },
#{Expression = "LocalPort"; Width = 10; Alignment = "Left" },
#{Expression = "RemoteAddress"; Width = 15 }, #{Expression = "RemotePort"; Width = 10; Alignment = "Left" },
#{Expression = "NameHost"; Width = 100 } |
Tee-Object -FilePath $logFile

Using two arrays to create registry keys/values

Trying to automate our font installation process for new PCs.
To install fonts, Windows adds the .ttf, .otf, etc. file to C:\Windows\Fonts and then creates a corresponding registry key in HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Fonts. A typical registry key would look like this:
Arial (TrueType) | Arial.ttf
To automate this, I've made two arrays using Get-ChildItem:
$names = Get-ChildItem -Path "C:\corp\install\fonts" | Select-Object name | Out-String | ForEach-Object {$_ -Replace "----","" ` -Replace "Name","" ` -Replace ".otf","" ` -Replace ".ttf","" } | ForEach-Object { $_.Trim() }
$files = Get-ChildItem -Path "C:\corp\install\fonts" | Select-Object name | Out-String | ForEach-Object {$_ -Replace "----","" ` -Replace "Name","" } | ForEach-Object { $_.Trim() }
Each $name in $names will be the name of the registry key, and each $file in $files will be the data for that registry key.
How would I go about doing this? I've attempted to use hash tables, PSObjects, nested ForEach loops, all to no avail. I have had difficulty finding anything on here and elsewhere that matches this situation exactly.
Error checking is not really necessary since there will always be a corresponding value.
REVISED FINAL SOLUTION:
Write-Host "Installing corporate fonts..."
Copy-Item -Path "C:\corp\install\fonts\*" -Destination "C:\Windows\Fonts" -Force -Recurse
$fontList = #()
$fonts = Get-ChildItem "C:\corp\install\fonts" | Select-Object -ExpandProperty Name
ForEach ( $font in $fonts ) {
$fontList += [PSCustomObject] #{
Name = $font -Replace ".otf","" ` -Replace ".ttf",""
File = $font
} |
ForEach-Object {
New-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Fonts" -Name $_.Name -Value $_.File
}
}
I must admit I don't fully understand your question so forgive me if this response is way off base, but it this what you're looking for? A table with both pieces of data in one?
Function CreateVariables {
$namevariables = #()
$filenames = ( Get-ChildItem "C:\corp\install\fonts" ).name
Foreach ( $name in $filenames ){
$namevariables += [PSCustomObject] #{
Name = $name -Replace "----","" ` -Replace "Name","" ` -Replace ".otf","" ` -Replace ".ttf",""
File = $name -Replace "----","" ` -Replace "Name",""
}
}
Return $namevariables
}
CreateVariables
Piping both name and value to set-itemproperty seem impossible. Foreach-object seems the way to go.
$path = 'hklm:\software\microsoft\windows nt\currentversion\fonts'
[pscustomobject]#{name='a name';value='a value'} |
foreach { set-itemproperty $path $_.name $_.value -whatif }
What if: Performing the operation "Set Property" on target "Item: HKEY_LOCAL_MACHINE\software\microsoft\windows nt\currentversion\fonts Property: a name".
You may prefer using this vbscript-like method to install fonts:
https://www.mondaiji.com/blog/other/it/10247-windows-install-fonts-via-command-line

Export csv via pscustomobject displaying incorrectly powershell

I have the following code:
function realtest
{
$files = Get-ChildItem -Path 'D:\data\' -Filter *.csv
$tester = [PSCustomObject]#{
foreach ($file in $files)
{
$tempName = $file.BaseName
$temp = Import-Csv $file
$tester | Add-Member -MemberType NoteProperty -Name $tempName -Value $temp.$tempName
}
$tester
$tester | Export-Csv "D:\result.csv" -NoTypeInformation
}
I am trying to export a bunch of data to CSV however when it is display the data on csv it is shown as below
"E0798T102","E0798T103"
"System.Object[]","System.Object[]"
but when i do it as a print on console it displays as the below
E0798T102 E0798T103
--------- ---------
{0, 0, 0, 0...} {0, 0, 0, 0...}
Ultimately, I want E0798T102 and E0798T103 as seperate columns in the result.csv
just to note, I will have 50 csv to loop through and each should display as its own column
Here is an incredibly inefficient answer to your question. If left as is, it assumes your CSV files already have a header with the CSV file basename:
$CSVs = Get-ChildItem -path 'D:\data\' -filter "*.csv" -file
$headers = $CSVs.basename
$table = [System.Data.DataTable]::new("Files")
foreach ($header in $headers) {
$table.Columns.Add($header) | out-null
}
foreach ($CSV in $CSVs) {
#$contents = Import-Csv $CSV -Header $CSV.basename # If CSV has no header
$contents = Import-Csv $CSV # If CSV contains header
$rowNumber = 0
foreach ($line in $Contents) {
$rowcount = $table.rows.count
if ($rowNumber -ge $rowCount) {
$row = $table.NewRow()
$row[$CSV.basename] = $line.$($CSV.basename)
$table.Rows.Add($row)
}
else {
$row = $table.rows[$rowNumber]
$row[$CSV.basename] = $line.$($CSV.basename)
}
$rowNumber++
}
}
$table | Export-Csv output.csv -NoTypeInformation
You can uncomment the commented $contents line if your CSV files do not have a header. You will just have to comment out the next $contents variable assignment if you uncomment the first.
Based on your snippet, this can be significantly simplified:
function Get-Csv {
$col = foreach ($file in Get-ChildItem -Path D:\data -Filter *.csv) {
$csv = Import-Csv -Path $file.FullName
[pscustomobject]#{
$csv.($file.BaseName) = $csv
}
}
$col | Export-Csv D:\result.csv -NoTypeInformation
return $col
}
However, a csv file seems like the wrong approach because you're trying to embed objects under a header. This doesn't really work in a tabular format as you only get one layer of depth. You should either expand all the properties on your objects, or use a different format that can represent depth, like json.
The reason for your formatting woes is due to how the serialization works. You're getting a string representation of your objects.
Converting to json isn't difficult, you just trade your Export-Csv call:
$col | ConvertTo-Json -Depth 100 | Out-File -FilePath D:\result.json
Note: I specify -Depth 100 because the cmdlet's default Depth is 2.

Export-Csv Cannot bind argument to parameter 'InputObject' because it is null

I am trying to fetch the AD group members information through power shell. But I am getting the messages as
Export-Csv : Cannot bind argument to parameter 'InputObject' because it is null.
Import-Module ActiveDirectory
$Path = Get-Location
$GroupName = Read-Host "Type the Group Name to check the members in it`t "
foreach ($group in $GroupName) {
$users = #{}
Get-ADUser -Filter '*' -Property Name, DisplayName | ForEach-Object {
$users[$_.DistinguishedName] = $_
}
$MemberCount = (Get-ADGroup $group -Properties Member | Select-Object -Expand Member).Count
Write-Host "`t Total Number of Users/Groups added on this Group : $MemberCount" -BackgroundColor DarkCyan
$Info = Get-ADGroup $group -Properties Member |
Select-Object -Expand Member |
ForEach-Object { $users[$_] }
$Info | Export-Csv -NoTypeInformation -Append -Force $Path\new.csv
The error means that $Info contains empty/null values. The most likely reason for their presence is that the group has members that aren't returned by Get-ADUser.
You can avoid the issue by checking for the presence of the key in the $users hashtable:
$Info = Get-ADGroup $group -Properties Member |
Select-Object -Expand Member |
ForEach-Object { if ($users.ContainsKey($_) {$users[$_]} }
If you want to further investigate the missing distinguished names you could collect them like this:
$missing = Get-ADGroup $group -Properties Member |
Select-Object -Expand Member |
ForEach-Object { if (-not $users.ContainsKey($_) {$_} }

local users and groups output to a file

I have a script that shows all the local users and their associated groups. However, I'm trying to output the results into a text file and that's where the script goes wrong, because it's not giving me the same results I'm receiving from the output window. For example, the code I have reads:
$LogFile = Test-Path C:\Users\FredAslami\Downloads\Test.txt
$LocalUsers = [ADSI]"WinNT://$env:COMPUTERNAME"
if ($LogFile) {
$LocalUsers.Children | where {$_.SchemaClassName -eq 'user'} | Foreach-Object {
$groups = $_.Groups() | Foreach-Object {
$_.GetType().InvokeMember("Name", 'GetProperty', $null, $_, $null)
}
$_ | Select-Object #{n='UserName';e={$_.Name}},
#{n='Groups';e={$groups -join ';'}}
}
Write-Host "Got User Groups Info"
Out-File -FilePath C:\Users\FredAslami\Downloads\Test.txt `
-InputObject $LocalUsers -Append
Write-Host "Added info to text"
}
$LocalUsers.Dispose()
When I run that the text in the file will read
distinguishedName :
Path : WinNT://R68-CUSTOM-01
I have also tried using Add-Content, but that doesn't work either. It will add something like:
System.DirectoryServices.DirectoryEntry
I also, tried to debug using Write-Host after it retrieves the local users and group info and another Write-Host after it writes the results into the text file and noticed that it's writing the results before it gathered all the info. So I tried using the Start-Sleep, and that didnt seem to work.
On the second line you have $LocalUsers = [ADSI]"WinNT://$env:COMPUTERNAME". You never assigned it a different value, so that's what you're seeing as your output.
I would recommend piping your Select-Object statement to Export-Csv. Much easier and cleaner.
You get different results in screen and file output, because you're doing different things with your data. The pipeline starting with $localUsers.Children builds a list of the user objects and their group memberships and echoes that to the screen, but you don't do anything else with that data. Instead you're writing the unmodified variable $localUsers to the output file.
If you want tabular data to go both to the console and a file, I'd suggest using Write-Host for the console output, and Export-Csv for the file output:
$LocalUsers.Children | where {$_.SchemaClassName -eq 'user'} | Foreach-Object {
$groups = $_.Groups() | Foreach-Object {
$_.GetType().InvokeMember('Name', 'GetProperty', $null, $_, $null)
}
$o = New-Object -Type PSObject -Property #{
'UserName' = $_.Name
'Groups' = $groups -join ';'
}
Write-Host $o
$o
} | Export-Csv 'C:\Users\FredAslami\Downloads\Test.txt' -NoType
If you want the output to go to the success output stream instead of the console, you could capture the result in a variable and output that in two different ways:
$users = $LocalUsers.Children | where {
$_.SchemaClassName -eq 'user'
} | Foreach-Object {
$groups = $_.Groups() | Foreach-Object {
$_.GetType().InvokeMember('Name', 'GetProperty', $null, $_, $null)
}
New-Object -Type PSObject -Property #{
'UserName' = $_.Name
'Groups' = $groups -join ';'
}
}
$users
$users | Export-Csv 'C:\Users\FredAslami\Downloads\Test.txt' -NoType

Resources