I'm attempting to add performance to a script that has an array of data of over 10,000 entries, then use it in a foreach-object statement to fill a blank ArrayList with new data by calling another function. I've been reading how I shouldn't use +=, which is how I learned, because the performance is dreadful as it tears down the array and rebuilds it for each item.
The issue I have is I need to call a function to fill an empty ArrayList, but I don't seem to be able to do this inside the .Add() method.
Old code:
Function get_gfe
Function get_os
$gfe = [System.Collections.ArrayList]#()
$gfe = get_gfe
$getos = [System.Collections.ArrayList]#()
$gfe | foreach { $getos += get_os $_}
This takes over an hour to fill $getos with the data.
I was hoping to use something like this instead, but it doesn't work, any help would be appreciated
$gfe | foreach { [void]$getos.Add(get_os $_)}
I know that you can use .Add($_), but that doesn't meet my needs and I couldn't find any references to using other code or calling functions inside the .Add()method.
Thanks in advance!
Why not expand the foreach-loop to something like this:
foreach ($entry in $gfe){
$os = get_os $entry
[void]$getos.add($os)
}
A foreach-loop also saves time compared to | piping into foreach-object.
Although of course since I don't know what your functions are actually doing, this could not be the most effective way to save time. You can determine that with measure-command.
Is it absolutely vital that $getos is of type System.Collections.ArrayList instead of a 'normal' array (System.Object[]) ?
If not, I think the next code could perform faster:
$getos = foreach ($entry in $gfe) {
get_os $entry # output the result of function get_os and collect in variable $getos
}
Thanks to all for the recommendations, they've helped me to gain a better understanding of foreach, arrays, and arrayLists. We've suspect the slowness is related to the foreach loop accessing a function, which uses an API for each serial number. We had recently upgrade our MDM console and swapped out the underlying hardware.
Related
so here is what I'm trying to accomplish.
I have a form for a new starter, New Starter Form.csv, that has the following headers and information:
firstname,lastname,teamname,startdate
Joe,Bloggs,Security Admin,01/01/18
I have a different csv called Team List.csv, that has the following headers and information:
teamlead,teamname,resgroup
A B,Marketing,RESMARKETING01G
C D,Product,RESPRODUCT01G
E F,Advertising,RESADVERTISING01G
G H,Security Admin,RESSECURITYADMIN01G
I want to import both CSV files into Powershell, run a comparisson that takes the team name from the New Starter Form, and checks if there are any matches in the Team List, and if so, add the relevant RES group to the new starter in AD.
Currently, I can import them, compare them, find a match, and find an index number for the record, but I'm struggling to the take this index number, and use it to get the relevant RES group. So far the code looks like this:
$teamlist = import-csv "\\location\Team List.csv"
$newstarter = import-csv "\\otherlocation\New Starter Form.csv"
[string]$teamname = Compare-Object -includeequal -excludedifferent -PassThru $newstarter.teamname $teamlist.teamname
$teamname
[array]::indexof($teamlist,$teamname)
And running that, provides us with this in the console, showing that we can indeed see the match, and that the matching record is the last (-1) one:
PS C:\WINDOWS\system32> $teamlist = import-csv "\\location\Team List.csv"
$newstarter = import-csv "\\otherlocation\New Starter Form.csv"
[string]$teamname = Compare-Object -includeequal -excludedifferent -PassThru $newstarter.teamname $teamlist.teamname
$teamname
[array]::indexof($teamlist,$teamname)
Security Administration
-1
I've not got a lot of experience with Powershell, and my coding knowledge is pretty limited overall, but I'm used to the concept that I can save the index value as a variable, and then I could call that variable back to do something like $teamlist.resgroup[VARIABLE HERE].
But if I try and declare a new variable before [array]::indexof($teamlist,$teamname), Powershell isn't happy.
Whilst I've not looked into it, I believe a possible alternative could be to add in a huge switch statement, but I may be looking at having 100+ teams overall, and I'd like to avoid inefficient code wherever I can. Am I missing something obvious though? Is there a better way (Or even just a functioning way would be great!) that this could work?
Any help you can provide would be greatly appreciated!
$teamlist = import-csv "\\location\Team List.csv"
$newstarter = import-csv "\\otherlocation\New Starter Form.csv"
# get a single new starter
$person = $newstarter | Where-Object { $_.firstname -eq 'Joe' -and $_.lastname -eq 'Bloggs' }
# get the new starters team
$team = $teamlist | Where-Object { $_.teamname -eq $person.teamname }
# get the new starters resource group
$resgroup = $team.resgroup
# use the resource group - this simply writes it to the console
Write-Host $resgroup
The code above will:
import your two csvs
grab a single new starter from your new starter csv, based on first &
last name
grab the team & resource group for that new starter from the team list
display the resgroup (this is where you will need to use to populate AD)
Just looking for someone to point me in the right direction please, repeatedly in PowerShell I come across collections that I can't get elements of using [x].
The one I've just hit is in IIS:
Import-Module WebAdministration
[System.Reflection.Assembly]::LoadFrom("C:\windows\system32\inetsrv\Microsoft.Web.Administration.dll") | Out-Null
$serverManager = New-Object Microsoft.Web.Administration.ServerManager
$site = $servermanager.sites[0]
This returns nothing. However, $servermanager.sites | foreach-object {$site = $_} correctly loops through each object.
Using gm only returns the members of the object, not the collection, and I've been unable to find anything online to explain this behaviour.
Actually managed to solve it just after finishing the question using gettype() then looking it up.
In this particular case you need to use $servermanager.sites.item(0) to get the item at index 0.
I've found some interesting behaviour in PowerShell Arrays, namely, if I declare an array as:
$array = #()
And then try to add items to it using the $array.Add("item") method, I receive the following error:
Exception calling "Add" with "1" argument(s): "Collection was of a fixed size."
However, if I append items using $array += "item", the item is accepted without a problem and the "fixed size" restriction doesn't seem to apply.
Why is this?
When using the $array.Add()-method, you're trying to add the element into the existing array. An array is a collection of fixed size, so you will receive an error because it can't be extended.
$array += $element creates a new array with the same elements as old one + the new item, and this new larger array replaces the old one in the $array-variable
You can use the += operator to add an element to an array. When you
use
it, Windows PowerShell actually creates a new array with the values of the
original array and the added value. For example, to add an element with a
value of 200 to the array in the $a variable, type:
$a += 200
Source: about_Arrays
+= is an expensive operation, so when you need to add many items you should try to add them in as few operations as possible, ex:
$arr = 1..3 #Array
$arr += (4..5) #Combine with another array in a single write-operation
$arr.Count
5
If that's not possible, consider using a more efficient collection like List or ArrayList (see the other answer).
If you want a dynamically sized array, then you should make a list. Not only will you get the .Add() functionality, but as #frode-f explains, dynamic arrays are more memory efficient and a better practice anyway.
And it's so easy to use.
Instead of your array declaration, try this:
$outItems = New-Object System.Collections.Generic.List[System.Object]
Adding items is simple.
$outItems.Add(1)
$outItems.Add("hi")
And if you really want an array when you're done, there's a function for that too.
$outItems.ToArray()
The most common idiom for creating an array without using the inefficient += is something like this, from the output of a loop:
$array = foreach($i in 1..10) {
$i
}
$array
Adding to a preexisting array:
[collections.arraylist]$array = 1..10
$array.add(11) > $null
I'm trying to get data out of an array by using following command:
$newarray = $current_ds2_data -match $codenumber
In this case the $current_ds2_data is an array as a result of the "Import-Csv" command and the $codenumber contains the value I want to search for in the array. This work OK.
Following is an example of the value of $newarray:
P_SYS_InternalName : #D_OCEV_ABC-
P_OCEV_Price : 0.15
P_NDS_ValidPN : 12345678
P_OCEV_PriceUnit :
P_NDS_VersionNumber : 1
Now I want to modify the value of the P_OCEV_Price field by doing
$newarray.P_OCEV_Price = 0.2
however this doesn't seem to work. It appears that $newarray.P_OCEV_Price contains no value. Somehow PS doesn't recognize P_OCEV_Price to be a cell of the array.
I also tried using
$newarray.["P_OCEV_Price"] = 0.2
to comply with hash-table formating
Next to this I tried defining the $newarray explicitly as an array or hash-table by using
$newarray = #()
or
$newarray = #{}
So far nothing seems to work. What am I doing wrong??
Since your $newarray variable is an array, you won't be able to use the simple $newarray.P_OCEV_Price syntax to change that value. Here are two alternate options that may help you, depending on your source data:
# Change the price of the first matching item
$newarray[0].P_OCEV_Price = 0.2
# Change the price for all matching items
$newarray | Foreach-Object { $_.P_OCEV_Price = 0.2 }
In cases like this, I usually like to point out that arrays of size 1 are easy to confuse with single objects in Powershell. If you try looking at $newarray and $newarray[0] with simple output statements the results will probably look identical. It's a good idea to keep GetType() handy in these cases.
# This will show a type of Object[]
$newarray.GetType()
# The type here will show PSCustomObject instead
($newarray[0]).GetType()
I've written a Powershell script which will compare two databases and come up with a list of objects in one of the databases to remove. I put those items (as a customized object with a name and schema) into an array.
In the next step of my script I iterate through the objects in the database and see if they match an object in my array. If I find a match then I go ahead and drop the object from my database. The problem that I ran into though, was that if I try to drop the object then the collection through which I'm iterating gets changed and I get an error message that the collection changed and IEnumerable won't work when that happens.
I tried to make a copy of collection, but I can't seem to stuff it into an array using the CopyTo method. Any suggestions?
My current code is below. When I run this the array $sprocs is empty.
function DropSQLObjects
{
param([object]$database, [object]$objectsToDrop)
$sprocs = #()
$database.StoredProcedures.CopyTo($sprocs, 0)
# If I do a $sprocs | out-host I see that the array is still empty
foreach ($objectToDrop in $objectsToDrop)
{
foreach ($sproc in $sprocs)
{
if ($sproc.Name -eq $objectToDrop.Name -and $sproc.Schema -eq $objectToDrop.Schema)
{
$sproc.Drop()
LogToSQL $database "Dropped Stored Procedure: $($objectToDrop.Schema).$($objectToDrop.Name)"
}
}
}
}
I'm adding this as an answer in case anyone else has need of this in the future. It turns out that I was really making things harder than they needed to be. Since I was using Powershell, the "where" function was better than iterating through the stored procedures.
Here's the code which solved my issue:
function DropSQLObjects
{
param([object]$database, [object]$objectsToDrop)
foreach ($objectToDrop in $objectsToDrop)
{
if ($database.StoredProcedures.Contains($objectToDrop.Name, $objectToDrop.Schema))
{
$sproc = $database.StoredProcedures | where {$_.Schema -eq $objectToDrop.Schema -and $_.Name -eq $objectToDrop.Name}
$sproc.Drop()
LogToSQL $database "Dropped Stored Procedure: $($objectToDrop.Schema).$($objectToDrop.Name)"
}
}
}
In actuality, I also have code to go against UserDefinedFunctions, but the code is mostly a cut-and-paste from the StoredProcedures portion.