Is there any nice way to ensure a pipeline result is always an array without the array literal #()?
Currently, I always found myself writing pipeline and assume that the result is an array, e.g.
$Results = $ResultFiles | Where Name -like $Pattern | Sort -Unique Name
# Processing that assume $Results is array, e.g.
Out-Host -InputObject "found $($Results.Length) matching files..."
# Further processing that assume $Results is array
Then, I realize that I need to ensure $Results is always an array. So, I come back to add the magic #(), #( in front and ) at the back:
$Results = #( <pipeline_statement> )
or even more magic by adding to an empty array
$Results = #() + <pipeline_statement>
My question: is there any way to ensure that a pipeline always results in an array? that doesn't requires the "magic" #()? I think of creating a function to collect pipeline results, e.g. ConvertTo-Array, like:
$Results = <pipeline_statement> | ConvertTo-Array
But I'd rather use default Cmdlet or idioms if any.
Note:
I also tempted to create ConvertTo-CustomObject as I often found myself creating PSCustomObject from hash table.
Yes, there is. There is an ability to directly typecast a single value to an array. If that value is already a [Object[]] nothing is effectively changed.
[Array]$result=<pipeline_statement>
As mentioned in comments by #wannabeprogrammer, this does not convert null value into an array. If something can return null, the following addition will remedy the situation:
if ($result -eq $null) { $result = #() }
Related
Is there a non for-loop way to remove some items from a arrayList?
$remotesumerrors = $remoteFiles | Select-String -Pattern '^[a-f0-9]{32}( )' -NotMatch
I want to remove the output of the above from the $remoteFiles var.. is there some pipe way to remove them?
Assuming all of the following:
you do need the results captured in $remotesumerrors separately
that $remoteFiles is a collection of System.IO.FileInfo instances, as output by Get-ChildItem, for instance
it is acceptable to save the result as an invariably new collection back to $remoteFiles,
you can use the .Where() array method as follows (this outperforms a pipeline-based solution based on the Where-Object cmdlet):
# Get the distinct set of the full paths of the files of origin
# from the Select-String results stored in $remotesumerrors
# as a hash set, which allows efficient lookup.
$errorFilePaths =
[System.Collections.Generic.HashSet[string]] $remotesumerrors.Path
# Get those file-info objects from $remoteFiles
# whose paths aren't in the list of the paths obtained above.
$remoteFiles = $remoteFiles.Where({ -not $errorFilePaths.Contains($_.FullName) })
As an aside:
Casting a collection to [System.Collections.Generic.HashSet[T]] is a fast and convenient way to get a set of distinct values (duplicates removed), but note that the resulting hash set's elements are invariably unordered and that, with strings, lookups are by default case-sensitive - see this answer for more information.
Use the Where-Object cmdlet to filter the list:
$remoteFiles = $remoteFiles |Where-Object { $_ |Select-String -Pattern '^[a-f0-9]{32}( )' -NotMatch }
If it truly was a [collections.arraylist], you could remove an element by value. There's also .RemoveAt(), to remove by array index.
[System.Collections.ArrayList]$array = 'a','b','c','d','e'
$array.remove
OverloadDefinitions
-------------------
void Remove(System.Object obj)
void IList.Remove(System.Object value)
$array.remove('c')
$array
a
b
d
e
Let assume that $remoteFiles is a file object of type System.IO.FileInfo. I also assume that you want to filter based on filename.
$remotesumerrors = $remoteFiles.name | Select-String -Pattern '^[a-f0-9]{32}' -NotMatch
What are trying to do with "( )" or what is query that you want to do.
edit: corrected answer based on comment
I've got a problem searching an INDEX in an array made up by query sessions command in a terminal server.
This is the problematic script:
# Array of logged users in terminal servers
$a=Get-RDUsersession -CollectionName "BLABLA" -ConnectionBroker BLABLA.BLA.BL
# Array of all users with two columns from active directory
$b=Get-ADUser -filter * -properties TelephoneNumber,SamAccountName
Now imagine logging in the terminal server using the account name TEST instead of test.
If I do:
$c = $b[$b.SamAccountName.indexof("test")].TelephoneNumber
then I don't get the telephone number.
I think that's because of the case sensitivity, isn't it? If I type TEST in the search command, I get the correct number.
Is there any simple way to solve this problem and make the search of the index case-insensitive?
I've read about using the method [StringComparison]"CurrentCultureIgnoreCase", but it seems not working with array.
Thanks.
Since $b is an Object[] type, then you would probably want to do a Where-Object.
$b | Where-Object -FilterScript {$_.Samaccountname -like '*Smith*'} | Select-Object -ExpandProperty 'telephoneNumber'
That being said, an array in Powershell can be indexed case-insensitively if it is converted to a [Collections.Generic.List[Object]] type.
$b = [Collections.Generic.List[Object]]$b
$b.FindIndex( {$args[0].sAMAccountName -eq 'test'} )
Note that pulling every single user object in AD and filtering using where-object or index matching can be very slow. You can instead Get-ADUser as needed or pull all ADusers using a filter that pulls only the users returned in $a.
If you insist on having all ADUsers in one spot with one pull, consider looping over the list once to make a hash lookup so you can easily index the hash value.
#Create account lookup hash
$accountlookup = #{}
foreach ($element in $accounts) {
$accountlookup[$element.SamAccountName] = $element
}
Hope that helps!
I have an array of custom objects in Powershell. Each object has a unique reference with which it can be looked up in the array and a value I wish to use later which is not necessarily unique. The array is populated in a for loop as below
#Create empty array
$array1 = #()
#Populate array
foreach($otherElement in $otherArray){
#Create custom array element
$arrayElement = New-Object PSObject
#Update Credential element
$arrayElement | Add-Member -type NoteProperty -Name 'objRef' -Value $($otherElement.getAttribute("ref") )
$arrayElement | Add-Member -type NoteProperty -Name 'objValue' -Value $($otherElement.getAttribute("someValue") )
#Add element to array
$array1 += $arrayElement
}
After building the array I want to get access to the objValue corresponding to the correct objRef in some way. I know that you can test if an array contains a value using the -contains parameter but I don't know how to get the index of the object with that value.
Basically this array is acting like a look up table. I want a function to put in an objRef to get out an objValue.
The objValue in this case is a System.Management.Automation.PSCredential, the password for each object is input at runtime for security. At work I have to sometimes install the same software 30 odd times on different machines with the same 5 credentials and figuring this out will help me automate the process reomtely.
Thanks in advance!
PowerShell is .NET based, so everything arrays in .NET can do, they can do in PowerShell. In particular, since they implement IList, they have the .IndexOf method: #(10, 4, 6).indexof(6) returns 2.
Of course, in your particular case a hash table is more appropriate, since looking up values there is constant time while searching through an array takes linear time. This matters little in the case of small arrays, but adds up quickly if you're doing things in a loop.
I'm looking for the negative intersection of two arrays. Each array has about 20k elements. I'm using a foreach loop over one array and looking each value up in the other array. I'm only keeping elements in the first array not found in the second array:
$deadpaths=#()
$ix=0
ForEach ($f in $FSBuildIDs)
{
if (-not($blArray -like $f)) {$deadpaths+=$paths[$ix]}
$ix++
}
$blArray contains valid IDs. $FSBuildIDs contains the IDs corresponding to the file system paths in $paths. The intent is to only keep the elements in $paths where the corresponding ID in $FSBuildIDS is NOT in $blArray.
Is there a better way to do this? The processing here takes an extremely long time. Both $blArray and $FSBuildIDs have about 20k elements and I suspect I'm looking at On^2 comparisons.
I thought about using a Dictionary with the elements of $FSBuildIDs as the keys and $paths as the values, but I can't figure out from the docs how to initialize and load the Dictionary (assuming this approach would speed things up). Obviously negative set intersection would be best but this isn't TSQL and I'm painfully aware that even V4 of PS doesn't support set operations.
Would using a dictionary in this problem speed up the comparisons? If so how do I create it from $FSBuildIDs and $paths? Any other techniques that might give me a performance boost vs. just iterating over these large(ish) lists?
Sample data for $blArray:
51012
51044
51049
51055
51058
51060
51073
51074
51077
51085
Sample data for $FSBuildIDs:
51001
51003
51005
51009
51013
51017
51018
51020
51021
51024
51026
Sample data for $paths:
\\server1\d$\software\anthill\var\artifacts\0000\3774\0000\3792\0005\2335
\\server1\d$\software\anthill\var\artifacts\0000\3774\0000\3792\0005\2336
\\server1\d$\software\anthill\var\artifacts\0000\3774\0000\3792\0005\2337
\\server1\d$\software\anthill\var\artifacts\0000\3774\0000\3792\0005\2338
\\server1\d$\software\anthill\var\artifacts\0000\3774\0000\3792\0005\2339
\\server1\d$\software\anthill\var\artifacts\0000\3774\0000\3792\0005\2340
\\server1\d$\software\anthill\var\artifacts\0000\3774\0000\3792\0005\2341
This is similar to the question posed previously, but different in some aspects. I'm essentially looking for guidance on constructing a dictionary from two existing arrays. I realized after posting that I really need a dictionary from $blarray as the keys and maybe $True as the value. The value is irrelevant. The important test is whether or not the current value in $FSBuildIDs is found in $blarray. That could be a dictionary lookup based on the ID as the key. That should speed up the processing, right?
I'm not clear on the comment that I'm destroying and recreating the array each time. Is that the $deadPaths array? Simply adding to it causes that? If so would I be better using a .Net ArrayList?
You could achieve a significant improvement by using the -contains operator instead of -like.
When the left-hand side of a -like operation is an array, PowerShell will iterate the array and perform a -like comparison against each and every entry.
-contains, on the other hand, returns as soon as a match is found.
Consider the following example:
$array1 = 1..2000
$array2 = 2..2001
$like = Measure-Command {
foreach($i in $array2){
$array1 -like $i
}
} |Select -Expand TotalMilliseconds
$contains = Measure-Command {
foreach($i in $array2){
$array1 -contains $i
}
} |Select -Expand TotalMilliseconds
Write-Host "Operation with -like took: $($like)ms"
Write-Host "Operation with -contains took: $($contains)ms"
Just like in your real-world example, we have 2 integer arrays with a large overlap. Let's see how it performs on my Windows 7 laptop (PowerShell 4.0):
I think the result speaks for itself :-)
That being said, you could, as you seem to anticipate, achieve an even greater improvement by populating a hashtable, using the values from the first array as keys:
$hashtable = $array1 |ForEach-Object -Begin {$t = #{}} -Process {
$t[$_] = $null
# the value doesn't matter, we're only interested in the key lookup
} -End { $t }
and then use the ContainsKey() method on the hashtable instead of -like:
foreach($i in $array2){
if($hashtable.ContainsKey($i)) { # do stuff }
}
You'll need to bump up the size of the array to see the actual difference (here using 20K items in the first array):
Final test script can be found here
I think this would be the start of what you are looking for. As discussed in comments we are going to do two comparisons. First to get the BuildID's we need to compare from from $FSBuildIDs and $blArray then we take the result of that to compare against the list of $paths. I am going to assume that it is just a string array of paths for now. Note there is room for error prevention and correction here. Still just testing for now.
$parsedIDs = Compare-Object $blArray $FSBuildIDs | Where{$_.SideIndicator -eq "=>"} | Select-Object -ExpandProperty InputObject
$paths = $paths | ForEach-Object{
$_ | Add-Member -MemberType NoteProperty -Name BuildID -Value (($_.Parent.Name + $_.Name) -as [int32]) -PassThru
}
$paths | Where-Object{$_.BuildID -in $parsedIDs}
First we compare the two ID arrays and keep the unique elements of $FSBuildIDs.
Next we go through the $paths. For each one we add a property that contains buildid. Where the buildid is the last two path elements concatenated and converted to an integer.
Once we have that a simple Where-Object give us the paths that have an id present from the first comparison.
To answer the question about building a hashtable:
$keyEnumerator = $FSBuildIDs.GetEnumerator()
$valEnumerator = $paths.GetEnumerator()
$idPathHash = #{}
foreach ($key in $keyEnumerator ) {
$null = $valEnumerator.movenext()
$idPathHash[$key] = $valEnumerator.current
}
Running this code on my system with a 20000 element array of fake data took 138ms.
To build the list of build ids not in the $idPathHash:
$buildIDsNotIn =
foreach ($buildId in $blArray) {
if (!$idPathHash.ContainsKey($buildId )) {
$buildId
}
}
This took 50ms on my system, with 20000 items in $blArray, again with fake data.
PS noob here (as will be obvious shortly) but trying hard to get better. In my exchange 2010 environment I import and export huge numbers of .pst files. Many will randomly fail to queue up and once they're not in the queue it's very tedious to sort through the source files to determine which ones need to be run again so I'm trying to write a script to do it.
first I run a dir on the list of pst files and fill a variable with the associated aliases of the accounts:
$vInputlist = dir $vPath -Filter *.pst |%{ get-mailbox -Identity $_.basename| select alias}
Then I fill a variable with the aliases of all the files/accounts that successfully queued:
$vBatch = foreach ($a in (Get-MailboxImportRequest -BatchName $vBatchname)) {get-mailbox $a.mailbox | select alias}
Then I compare the two arrays to see which files I need to queue up again:
foreach($should in $vInputlist){if ($vBatch -notcontains $should){Write-Host $should ""}}
It seems simple enough yet the values in the arrays never match, or not match, as the case may be. I've tried both -contains and -notcontains. I have put in a few sanity checks along the way like exporting the variables to the screen and/or to csv files and the data looks fine.
For instance, when $vInputlist is first filled I send it to the screen and it looks like this:
Alias
MapiEnableTester1.psiloveyou.com
MapiEnableTester2.psiloveyou.com
MapiEnableTester3.psiloveyou.com
MapiEnableTester4.psiloveyou.com
Yet that last line of code I displayed above (..write-host $should,"") will output this:
#{Alias=MapiEnableTester1.psiloveyou.com}
#{Alias=MapiEnableTester2.psiloveyou.com}
#{Alias=MapiEnableTester3.psiloveyou.com}
#{Alias=MapiEnableTester4.psiloveyou.com}
(those all display as a column, not sure why they won't show that way here)
I've tried declaring the arrays like this, $vInputlist = #()
I've tried instead of searching for the alias just cleaning .pst off off the $_.basename using .replace
I've searched on comparing arrays til I'm blue in the fingers and I don't think my comparison is wrong, I believe that somehow no matter how I fill these variables I am corrupting or changing the data so that seemingly matching data simply doesn't.
Any help would be greatly appreciated. TIA
Using -contains to compare objects aren't easy because the objects are never identical even though they have the same property with the same value. When you use select alias you get an array of pscustomobjects with the property alias.
Try using the -expand parameter in select, like
select -expand alias
Using -expand will extract the value of the alias property, and your lists will be two arrays of strings instead, which can be compared using -contains and -notcontains.
UPDATE I've added a sample to show you what happends with your code.
#I'm creating objects that are EQUAL to the ones you have in your code
#This will simulate the objects that get through the "$vbatch -notcontains $should" test
PS > $arr = #()
PS > $arr += New-Object psobject -Property #{ Alias="MapiEnableTester1.psiloveyou.com" }
PS > $arr += New-Object psobject -Property #{ Alias="MapiEnableTester2.psiloveyou.com" }
PS > $arr += New-Object psobject -Property #{ Alias="MapiEnableTester3.psiloveyou.com" }
PS > $arr | ForEach-Object { Write-Host $_ }
#{Alias=MapiEnableTester1.psiloveyou.com}
#{Alias=MapiEnableTester2.psiloveyou.com}
#{Alias=MapiEnableTester3.psiloveyou.com}
#Now this is what you will get if you use "... | select -expand alias" instead of "... | select alias"
PS > $arrWithExpand = $arr | select -expand alias
PS > $arrWithExpand | ForEach-Object { Write-Host $_ }
MapiEnableTester1.psiloveyou.com
MapiEnableTester2.psiloveyou.com
MapiEnableTester3.psiloveyou.com