How can I repeat an iteration in a PowerShell foreach loop? - loops

I frequently have to loop through sets of data (SQL results, lists of computers, etc.) in PowerShell, performing the same operation (e.g function X) on each time. I use foreach loops almost exclusively as they are simple, effective and easily understood by others who may need to follow my code.
I would like to make some of my code more robust, in the sense of retrying operations that fail (up to Y times). There's more than one way to achieve this, for example within the foreach loop, wrapping function X in a do{} while() loop. In this example, assume that function X only returns non-null results when it is "successful":
foreach($dataitem in $dataset){
$Result = $null
$Attempts = 0
do{
$Attempts++
$Result = <call function X on $dataitem>
} while(($Attempts -lt 3) -and (-not $Result))
}
I was wondering whether there was any way to flatten this logic a bit, i.e. a more advanced way of using foreach loops, so I can do away with the do{} while() loop.
I have encountered the opposite of what I want, namely using $foreach.MoveNext() to skip forwards in the loop, but haven't found anything that (dangerously?) would keep foreach processing the same item.
Essentially: Can a foreach loop be made to re-do an iteration?.

I wouldn't try to re-do an iteration of a foreach loop. That doesn't feels right to me. Instead, I would create a function that implement something like a retry-logic pattern:
function Retry-Process()
{
Param(
[scriptblock]$action,
[scriptBlock]$validator,
[int]$retryCount
)
1 .. $retryCount | % {
$result = & $action
if (& $validator ($result))
{
$result
break
}
}
}
Example call:
Retry-Process -action { '3' } -validator { Param($a) $a -eq '3'} -retryCount 5
And within your foreach loop:
foreach($dataitem in $dataset) {
$Result = Retry-Process `
-action { <call function X on $dataitem> } `
-validator { Param($returnValue) $returnValue } ` # Validate return value != null
-retryCount 3
}

While I agree with Jisaak, to answer your question you could achieve this using something like the following
foreach ($i in 1..10) {
$i
if ($limit++ -gt 10) { break } # just to stop infinite loop
if ($i -eq 4) {
foreach.Reset()
1..($i-1) | Foreach-Object {
[void]$foreach.MoveNext()
}
}
}
Of course I wouldn't do anything this silly outside of an exercise. The mix of foreach() {} and Foreach-Object is done to simplify the scope of the automatic IEnumeration object $foreach.
Edit
I thought it useful to include a link to a page that describes the difference between foreach as a keyword and as a cmdlet:
http://www.powershelladmin.com/wiki/PowerShell_foreach_loops_and_ForEach-Object
about_Foreach
Foreach-Object

Related

String comparison in PowerShell doesn't seem to work

I am trying to compare strings from one array to every string from the other. The current method works, as I have tested it with simpler variables and it retrieves them correctly. However, the strings that I need it to compare don't seem to be affected by the comparison.
The method is the following:
$counter = 0
for ($num1 = 0; $num1 -le $serverid_oc_arr.Length; $num1++) {
for ($num2 = 0; $num2 -le $moss_serverid_arr.Length; $num2++) {
if ($serverid_oc_arr[$num1] -eq $moss_serverid_arr[$num2]) {
break
}
else {
$counter += 1
}
if ($counter -eq $moss_serverid_arr.Length) {
$unmatching_serverids += $serverid_oc_arr[$num1]
$counter = 0
break
}
}
}
For each string in the first array it is iterating between all strings in the second and comparing them. If it locates equality, it breaks and goes to the next string in the first array. If it doesn't, for each inequality it is adding to the counter and whenever the counter hits the length of the second array (meaning no equality has been located in the second array) it adds the corresponding string to a third array that is supposed to retrieve all strings that don't match to anything in the second array in the end. Then the counter is set to 0 again and it breaks so that it can go to the next string from the first array.
This is all okay in theory and also works in practice with simpler strings, however the strings that I need it to work with are server IDs and look like this:
289101b4-3e6c-4495-9c67-f317589ba92c
Hence, the script seems to completely ignore the comparison and just puts all the strings from the first array into the third one and retrieves them at the end (sometimes also some random strings from both first and second array).
Another method I tried with similar results was:
$unmatching_serverids = $serverid_oc_arr | Where {$moss_serverid_arr -NotContains $_}
Can anyone spot any mistake that I may be making anywhere?
The issue with your code is mainly the use of -le instead of -lt, collection index starts at 0 and the collection's Length or Count property starts at 1. This was causing $null values being added to your result collection.
In addition to the above, $counter was never getting reset if the following condition was not met:
if ($counter -eq $moss_serverid_arr.Length) { ... }
You would need to a add a $counter = 0 outside inner for loop to prevent this.
Below you can find the same, a little bit improved, algorithm in addition to a test case that proves it's working.
$ran = [random]::new()
$ref = [System.Collections.Generic.HashSet[int]]::new()
# generate a 100 GUID collection
$arr1 = 1..100 | ForEach-Object { [guid]::NewGuid() }
# pick 90 unique GUIDs from arr1
$arr2 = 1..90 | ForEach-Object {
do {
$i = $ran.Next($arr1.Count)
} until($ref.Add($i))
$arr1[$i]
}
$result = foreach($i in $arr1) {
$found = foreach($z in $arr2) {
if ($i -eq $z) {
$true; break
}
}
if(-not $found) { $i }
}
$result.Count -eq 10 # => Must be `$true`
As side, the above could be reduced to this using the .ExceptWith(..) method from HashSet<T> Class:
$hash = [System.Collections.Generic.HashSet[guid]]::new([guid[]]$arr1)
$hash.ExceptWith([guid[]]$arr2)
$hash.Count -eq 10 # => `$true`
The working answer that I found for this is the below:
$unmatching_serverids = #()
foreach ($item in $serverid_oc_arr)
{
if ($moss_serverid_arr -NotContains $item)
{
$unmatching_serverids += $item
}
}
No obvious differences can be seen between it and the other methods (especially for the for-loop, this is just a simplified variant), but somehow this works correctly.

Perl Syntax combining `foreach` and `if`: Shouldn't it work, i.e.: Why doesn't it work?

Trying to get the maximum in an array that cannot contain negative numbers I tried:
my #v;
#...
my $max = 0;
$max = $_
if ($_ > $max)
foreach (#v);
I get a syntax error with perl 5.18.2.
However (1) statement($_) foreach (#v); and (2) $max = $_ if ($_ > $max); are both OK, and they do what they are supposed to do.
So if (1) and (2) are both valid statements, why can't they be combined in the pattern used for (1)?
Just that noone needs to suggest that; here's how I solved the problem using a different syntax:
foreach (#v) {
$max = $_
if ($_ > $max);
}
So if (1) and (2) are both valid statements, why can't they be combined in the pattern used for (1)?
The postfix syntax (aka statement modifier), which you are using here, just does not support this kind of statement.
The documentation says (emphasis mine):
Any simple statement may optionally be followed by a SINGLE modifier, just before the terminating semicolon (or block ending).
As for your task itself: getting the max value from an array is a very common requirement. A simple and standard approach uses function max() from core module List::Util:
use List::Util qw/max/;
my #v;
my $max = max 0, #v;
I will add this more generic answer, because wanting to combine for & if statement modifiers is pretty common, not only for trivial cases where you can use a (much faster) CPAN module.
The reason why you might want to not do the standard foreach that uses a block, is that a block costs time, which might be insignificant if what happens inside the loop is slow, but might be over 10% of overhead if you are doing something like a simple assignment. e.g. this simple example:
my #v = map { rand } ( 1..100 );
cmpthese(-2, {
postfix => sub {
my $last;
$last = $_ for #v;
},
block => sub {
my $last;
foreach (#v) {$last = $_}
},
});
Gives me:
Rate block postfix
block 214369/s -- -11%
postfix 241751/s 13% --
So... while you cannot use two statement modifiers (postfix if, for), most of the time you can replace the for with a map, retaining the speed of a block-less loop. For example OP's example would be map {$max = $_ if $_ > $max} #v and benchmarked:
my #v = map { rand } ( 1..100 );
cmpthese(-2, {
map => sub {
my $max = 0;
map {$max = $_ if $_ > $max} #v;
},
foreach => sub {
my $max = 0;
foreach (#v) {$max = $_ if $_ > $max}
},
});
gives:
Rate foreach map
foreach 205585/s -- -13%
map 236303/s 15% --
As I said, in the specific case, you should definitely use List::Util which is written in C and will be an order of magnitude faster...

Why does my break statement break "too far"?

Here's a snippet of my code (based on this previous question):
$rgdNames = (Get-AzureRmResourceGroupDeployment -ResourceGroupName "$azureResGrpName").DeploymentName
$siblings = $rgdNames | Where-Object{$_ -match "^($hostname)(\d+)$" }
if ($siblings) {
# Make a list of all numbers in the list of hostnames
$serials = #()
foreach ($sibling in $siblings) {
# $sibling -split ($sibling -split '\d+$') < split all digits from end, then strip off everything at the front
# Then convert it to a number and add that to $serials
$hostnumber = [convert]::ToInt32([string]$($sibling -split ($sibling -split '\d+$'))[1], 10)
$serials += $hostnumber
}
(1..$siblingsMax) | foreach { # Iterate over all valid serial numbers
if (!$serials.Contains($_)) { # Stop when we find a serial number that isn't in the list of existing hosts
$serial = $_
# break # FIXME: For some reason, this break statement breaks "too far"
}
}
} else {
$serial = 1
}
write-output("serial = $serial") # does not print
# ...more statements here, but they're never called :(
I have been looking at it for a while, but can't figure out why the break statement (if un-commented) stops my program instead of just breaking out of its foreach loop. Is there something about foreach that I'm not aware of, or does break just work differently than it would in Java?
For the time being, I'm working around this using an extra if test, and it doesn't mean (too) much that the loop runs its entire length. But it's ugly!
This construct:
(1..$siblingsMax) | foreach { # Iterate over all valid serial numbers
# do stuff
}
is NOT a foreach loop - it's a pipeline with a call to the ForEach-Object cmdlet (aliased as foreach) - and keywords designed to interrupt the control flow of a loop (like break or continue) will NOT act in the same way in these two different cases.
Use a proper foreach loop and break will behave as you expect:
foreach($i in 1..$siblingsMax){
# do stuff
if($shouldBreak)
{
break
}
}
Alternatively, you can abuse the fact that continue behaves like you would expect break in a ForEach-Object process block:
(1..$siblingsMax) | foreach { # Iterate over all valid serial numbers
if($shouldBreak)
{
continue
}
}
although I would strongly discourage this approach (it'll only lead to more confusion)

Powershell: multidimensional array changes return value from "system.object[]"

I wondered if anyone could shed some light on the issue I am facing when returning values from a multidimensional array through a function:
$ArrayList = #()
function MultiDimensionalArrayTest
{
param(
[array]$ArrayList
)
for($i = 0; $i -lt 1; $i++)
{
$ArrayModify += ,#("Test", "Test")
}
return $ArrayModify
}
$ArrayModify = MultiDimensionalArrayTest
foreach ($item in $ArrayModify)
{
Write-Host $item[0]
}
When the loop is executed once the values returned are:
T
T
However if the for statement is lopped twice, the values returned are:
Test
Test
My aim is to retrieve x amount of values "Test" from the Write-Host $item[0] regardless of how many times the statement is executed
It appears that if two or more rows are captured and returned in the $ArrayModify array, the value is a system.object[], however if looped once, the value "Test, Test" is captured and when Write-Host $item[0] is executed, it will print T.
Any advice would be greatly appreciated.
Not the cleanest way of dealing with it but you need to prevent PowerShell from unrolling the single element array into an array with two elements.
function MultiDimensionalArrayTest{
$ArrayModify = #()
for($i = 0; $i -lt 1; $i++){
$ArrayModify += ,#("Test$i", "Test$i$i")
}
,#($ArrayModify)
}
Using the above function will get the desired output I believe. ,#($ArrayModify) ensures that the array is returned and not unrolled into its elements as you saw above.
$ArrayList = #()
$ArrayList = MultiDimensionalArrayTest
foreach ($item in $ArrayList){$item[0]}
Giving the output for $i -lt 1 in the loop
Test0
Giving the output for $i -lt 2 in the loop
Test0
Test1
Your Output
Concerning your output from your example with $i -lt 1 PowerShell is unrolling the array into a single dimension array with 2 elements "Test" and "Test". You are seeing the letter "T" since all strings support array indexing and will return the character from the requested position.
Other issues with code
Not to beat a dead horse but really look at the other answers and comments as they provide some tips as to some coding errors and anomalies of the code you presented in the question.
First of all I notice several mistakes in your code.
This line $ArrayList = #() is useless since you don't use the $ArrayList variable afterwards.
Regarding the MultiDimensionalArrayTest function, you declared an $ArrayList argument but you never use it in the function.
Finally when this line makes no sense $ArrayModify = MultiDimensionalArrayTest = $item, you probably meant $ArrayModify = MultiDimensionalArrayTest.
This is not a complete answer to your question, since I am not sure what you are trying to achieve, but take a look at this line:
$ArrayModify = MultiDimensionalArrayTest = $item
This makes no sense, since you are calling the function MultiDimensionalArrayTest and passing two arguments to it, which are "=" (powershell assumes this is a string) and $item (null object). Then you assign whatever is returned to $ArrayModify.
The reason "T" is outputted, is because you are outputting the first element of what is at the moment a string. "Test" is outputted when $item is an array of strings.

Array.Find and IndexOf for multiple elements that are exactly the same object

I have trouble of getting index of the current element for multiple elements that are exactly the same object:
$b = "A","D","B","D","C","E","D","F"
$b | ? { $_ -contains "D" }
Alternative version:
$b = "A","D","B","D","C","E","D","F"
[Array]::FindAll($b, [Predicate[String]]{ $args[0] -contains "D" })
This will return:
D
D
D
But this code:
$b | % { $b.IndexOf("D") }
Alternative version:
[Array]::FindAll($b, [Predicate[String]]{ $args[0] -contains "D" }) | % { $b.IndexOf($_) }
Returns:
1
1
1
so it's pointing at the index of the first element. How to get indexes of the other elements?
You can do this:
$b = "A","D","B","D","C","E","D","F"
(0..($b.Count-1)) | where {$b[$_] -eq 'D'}
1
3
6
mjolinor's answer is conceptually elegant, but slow with large arrays, presumably due to having to build a parallel array of indices first (which is also memory-inefficient).
It is conceptually similar to the following LINQ-based solution (PSv3+), which is more memory-efficient and about twice as fast, but still slow:
$arr = 'A','D','B','D','C','E','D','F'
[Linq.Enumerable]::Where(
[Linq.Enumerable]::Range(0, $arr.Length),
[Func[int, bool]] { param($i) $arr[$i] -eq 'D' }
)
While any PowerShell looping solution is ultimately slow compared to a compiled language, the following alternative, while more verbose, is still much faster with large arrays:
PS C:\> & { param($arr, $val)
$i = 0
foreach ($el in $arr) { if ($el -eq $val) { $i } ++$i }
} ('A','D','B','D','C','E','D','F') 'D'
1
3
6
Note:
Perhaps surprisingly, this solution is even faster than Matt's solution, which calls [array]::IndexOf() in a loop instead of enumerating all elements.
Use of a script block (invoked with call operator & and arguments), while not strictly necessary, is used to prevent polluting the enclosing scope with helper variable $i.
The foreach statement is faster than the Foreach-Object cmdlet (whose built-in aliases are % and, confusingly, also foreach).
Simply (implicitly) outputting $i for each match makes PowerShell collect multiple results in an array.
If only one index is found, you'll get a scalar [int] instance instead; wrap the whole command in #(...) to ensure that you always get an array.
While $i by itself outputs the value of $i, ++$i by design does NOT (though you could use (++$i) to achieve that, if needed).
Unlike Array.IndexOf(), PowerShell's -eq operator is case-insensitive by default; for case-sensitivity, use -ceq instead.
It's easy to turn the above into a (simple) function (note that the parameters are purposely untyped, for flexibility):
function get-IndicesOf($Array, $Value) {
$i = 0
foreach ($el in $Array) {
if ($el -eq $Value) { $i }
++$i
}
}
# Sample call
PS C:\> get-IndicesOf ('A','D','B','D','C','E','D','F') 'D'
1
3
6
You would still need to loop with the static methods from [array] but if you are still curious something like this would work.
$b = "A","D","B","D","C","E","D","F"
$results = #()
$singleIndex = -1
Do{
$singleIndex = [array]::IndexOf($b,"D",$singleIndex + 1)
If($singleIndex -ge 0){$results += $singleIndex}
}While($singleIndex -ge 0)
$results
1
3
6
Loop until a match is not found. Assume the match at first by assigning the $singleIndex to -1 ( Which is what a non match would return). When a match is found add the index to a results array.

Resources