Select-String bug splitting up lines [duplicate] - arrays

I'm using Powershell to set up IIS bindings on a web server, and having a problem with the following code:
$serverIps = gwmi Win32_NetworkAdapterConfiguration
| Where { $_.IPAddress }
| Select -Expand IPAddress
| Where { $_ -like '*.*.*.*' }
| Sort
if ($serverIps.length -le 1) {
Write-Host "You need at least 2 IP addresses for this to work!"
exit
}
$primaryIp = $serverIps[0]
$secondaryIp = $serverIps[1]
If there's 2+ IPs on the server, fine - Powershell returns an array, and I can query the array length and extract the first and second addresses just fine.
Problem is - if there's only one IP, Powershell doesn't return a one-element array, it returns the IP address (as a string, like "192.168.0.100") - the string has a .length property, it's greater than 1, so the test passes, and I end up with the first two characters in the string, instead of the first two IP addresses in the collection.
How can I either force Powershell to return a one-element collection, or alternatively determine whether the returned "thing" is an object rather than a collection?

Define the variable as an array in one of two ways...
Wrap your piped commands in parentheses with an # at the beginning:
$serverIps = #(gwmi Win32_NetworkAdapterConfiguration
| Where { $_.IPAddress }
| Select -Expand IPAddress
| Where { $_ -like '*.*.*.*' }
| Sort)
Specify the data type of the variable as an array:
[array]$serverIps = gwmi Win32_NetworkAdapterConfiguration
| Where { $_.IPAddress }
| Select -Expand IPAddress
| Where { $_ -like '*.*.*.*' }
| Sort
Or, check the data type of the variable...
IF ($ServerIps -isnot [array])
{ <error message> }
ELSE
{ <proceed> }

Force the result to an Array so you could have a Count property. Single objects (scalar) do not have a Count property. Strings have a length property so you might get false results, use the Count property:
if (#($serverIps).Count -le 1)...
By the way, instead of using a wildcard that can also match strings, use the -as operator:
[array]$serverIps = gwmi Win32_NetworkAdapterConfiguration -filter "IPEnabled=TRUE" | Select-Object -ExpandProperty IPAddress | Where-Object {($_ -as [ipaddress]).AddressFamily -eq 'InterNetwork'}

You can either add a comma(,) before return list like return ,$list or cast it [Array] or [YourType[]] at where you tend to use the list.

If you declare the variable as an array ahead of time, you can add elements to it - even if it is just one...
This should work...
$serverIps = #()
gwmi Win32_NetworkAdapterConfiguration
| Where { $_.IPAddress }
| Select -Expand IPAddress
| Where { $_ -like '*.*.*.*' }
| Sort | ForEach-Object{$serverIps += $_}

You can use Measure-Object to get the actual object count, without resorting to an object's Count property.
$serverIps = gwmi Win32_NetworkAdapterConfiguration
| Where { $_.IPAddress }
| Select -Expand IPAddress
| Where { $_ -like '*.*.*.*' }
| Sort
if (($serverIps | Measure).Count -le 1) {
Write-Host "You need at least 2 IP addresses for this to work!"
exit
}

Return as a referenced object, so it never converted while passing.
return #{ Value = #("single data") }

I had this problem passing an array to an Azure deployment template. If there was one object, PowerShell "converted" it to a string. In the example below, $a is returned from a function that gets VM objected according to the value of a tag. I pass the $a to the New-AzureRmResourceGroupDeployment cmdlet by wrapping it in #(). Like so:
$TemplateParameterObject=#{
VMObject=#($a)
}
New-AzureRmResourceGroupDeployment -ResourceGroupName $RG -Name "TestVmByRole" -Mode Incremental -DeploymentDebugLogLevel All -TemplateFile $templatePath -TemplateParameterObject $TemplateParameterObject -verbose
VMObject is one of the template's parameters.
Might not be the most technical / robust way to do it, but it's enough for Azure.
Update
Well the above did work. I've tried all the above and some, but the only way I have managed to pass $vmObject as an array, compatible with the deployment template, with one element is as follows (I expect MS have been playing again (this was a report and fixed bug in 2015)):
[void][System.Reflection.Assembly]::LoadWithPartialName("System.Web.Extensions")
foreach($vmObject in $vmObjects)
{
#$vmTemplateObject = $vmObject
$asJson = (ConvertTo-Json -InputObject $vmObject -Depth 10 -Verbose) #-replace '\s',''
$DeserializedJson = (New-Object -TypeName System.Web.Script.Serialization.JavaScriptSerializer -Property #{MaxJsonLength=67108864}).DeserializeObject($asJson)
}
$vmObjects is the output of Get-AzureRmVM.
I pass $DeserializedJson to the deployment template' parameter (of type array).
For reference, the lovely error New-AzureRmResourceGroupDeployment throws is
"The template output '{output_name}' is not valid: The language expression property 'Microsoft.WindowsAzure.ResourceStack.Frontdoor.Expression.Expressions.JTokenExpression'
can't be evaluated.."

There is a way to deal with your situation. Leave most of you code as-is, just change the way to deal with the $serverIps object. This code can deal with $null, only one item, and many items.
$serverIps = gwmi Win32_NetworkAdapterConfiguration
| Where { $_.IPAddress }
| Select -Expand IPAddress
| Where { $_ -like '*.*.*.*' }
| Sort
# Always use ".Count" instead of ".Length".
# This works on $null, only one item, or many items.
if ($serverIps.Count -le 1) {
Write-Host "You need at least 2 IP addresses for this to work!"
exit
}
# Always use foreach on a array-possible object, so that
# you don't have deal with this issue anymore.
$serverIps | foreach {
# The $serverIps could be $null. Even $null can loop once.
# So we need to skip the $null condition.
if ($_ -ne $null) {
# Get the index of the array.
# The #($serverIps) make sure it must be an array.
$idx = #($serverIps).IndexOf($item)
if ($idx -eq 0) { $primaryIp = $_ }
if ($idx -eq 1) { $secondaryIp = $_ }
}
}
In PowerShell Core, there is a .Count property exists on every objects. In Windows PowerShell, there are "almost" every object has an .Count property.

Related

Query PSCustomObject Array for row with largest value

I'm trying to find the row with an attribute that is larger than the other row's attributes. Example:
$Array
Name Value
---- ----
test1 105
test2 101
test3 512 <--- Selects this row as it is the largest value
Here is my attempt to '1 line' this but It doesn't work.
$Array | % { If($_.value -gt $Array[0..($Array.Count)].value){write-host "$_.name is the largest row"}}
Currently it outputs nothing.
Desired Output:
"test1 is the largest row"
I'm having trouble visualizing how to do this efficiently with out some serious spaghetti code.
You could take advantage of Sort-Object to rank them by the property "Value" like this
$array = #(
[PSCustomObject]#{Name='test1';Value=105}
[PSCustomObject]#{Name='test2';Value=101}
[PSCustomObject]#{Name='test3';Value=512}
)
$array | Sort-Object -Property value -Descending | Select-Object -First 1
Output
Name Value
---- -----
test3 512
To incorporate your write host you can just run the one you select through a foreach.
$array | Sort-Object -Property value -Descending |
Select-Object -First 1 | Foreach-Object {Write-host $_.name,"has the highest value"}
test3 has the highest value
Or capture to a variable
$Largest = $array | Sort-Object -Property value -Descending | Select-Object -First 1
Write-host $Largest.name,"has the highest value"
test3 has the highest value
PowerShell has many built in features to make tasks like this easier.
If this is really an array of PSCustomObjects you can do something like:
$Array =
#(
[PSCustomObject]#{ Name = 'test1'; Value = 105 }
[PSCustomObject]#{ Name = 'test2'; Value = 101 }
[PSCustomObject]#{ Name = 'test3'; Value = 512 }
)
$Largest = ($Array | Sort-Object Value)[-1].Name
Write-host $Largest,"has the highest value"
This will sort your array according to the Value property. Then reference the last element using the [-1] syntax, then return the name property of that object.
Or if you're a purist you can assign the variable like:
$Largest = $Array | Sort-Object Value | Select-Object -Last 1 -ExpandProperty Name
If you want the whole object just remove .Name & -ExpandProperty Name respectively.
Update:
As noted PowerShell has some great tools to help with common tasks like sorting & selecting data. However, that doesn't mean there's never a need for looping constructs. So, I wanted to make a couple of points about the OP's own answer.
First, if you do need to reference array elements by index use a traditional For loop, which might look something like:
For( $i = 0; $i -lt $Array.Count; ++$i )
{
If( $array[$i].Value -gt $LargestValue )
{
$LargestName = $array[$i].Name
$LargestValue = $array[$i].Value
}
}
$i is commonly used as an iteration variable, and within the script block is used as the array index.
Second, even the traditional loop is unnecessary in this case. You can stick with the ForEach loop and track the largest value as and when it's encountered. That might look something like:
ForEach( $Row in $array )
{
If( $Row.Value -gt $LargestValue )
{
$LargestName = $Row.Name
$LargestValue = $Row.Value
}
}
Strictly speaking you don't need to assign the variables beforehand, though it may be a good practice to precede either of these with:
$LargestName = ""
$LargestValue = 0
In these examples you'd have to follow with a slightly modified Write-Host command
Write-host $LargestName,"has the highest value"
Note: Borrowed some of the test code from Doug Maurer's Fine Answer. Considering our answers were similar, this was just to make my examples more clear to the question and easier to test.
Figured it out, hopefully this isn't awful:
$Count = 1
$CurrentLargest = 0
Foreach($Row in $Array) {
# Compare This iteration vs the next to find the largest
If($Row.value -gt $Array.Value[$Count]){$CurrentLargest = $Row}
Else {$CurrentLargest = $Array[$Count]}
# Replace the existing largest value with the new one if it is larger than it.
If($CurrentLargest.Value -gt $Largest.Value){ $Largest = $CurrentLargest }
$Count += 1
}
Write-host $Largest.name,"has the highest value"
Edit: its awful, look at the other answers for a better way.

How do i compare every PC name from a multiobject array to a single object array

Hi Powershell newbie alert,
I am trying to compare the computer name of my $getADComp function (CN) with the output of $WSUSArr.
I want to know which PC's are in the WSUS but not and the AD and vice versa. I want the PC names to go in 2 diffrent result arrays so i can use these again later
Eventually i want something like this
$separator = "."
$GetWSUSComp = Get-wsuscomputer -UpdateServer $wsus
$GetADComp = Get-ADComputer -Filter * -Property CN, CanonicalName, Description | Select-Object CN, CanonicalName, Description
$WSUSArr = #()
for ($i = 0; $i -lt $GetWSUSComp.Count; $i++){
$WSUSArr += $GetWSUSComp[$i].FullDomainName.split($separator)[0].ToUpper()
}
Compare-Object -ReferenceObject $WSUSArr -DifferenceObject $GetADComp
If there is a more efficient way to do this feel free to make use of another method (hash table, etc.)
I have another array which is called $WSUSArr which contains the names of all the computer that are connected to the WSUS server i will have to compare those two lists with eachother.
Why not go for an array of objects ?
$GetADComp = Get-ADComputer -Filter * -Property CN, DistinguishedName, Description |
Select-Object CN, DistinguishedName, Description
This way you avoid gathering all properties with -Property * where you only want three.
Every property of items in the array can be accessed by using
$GetADComp[$index].CN
$GetADComp[$index].DistinguishedName
$GetADComp[$index].Description
and compared with an array of CN's like
$GetADComp | Where-Object { $_.CN -eq $WSUSArr[$index] }
In order to compare the computer CN's with the objects returned from the Get-ADComputer cmdlet:
1.
Get the computers that are both in AD and in the WSUS array.
If you want this to be a simple string array of just the CN's, do this:
$ADcomputersInWsus = $GetADComp | Where-Object { $WSUSArr -contains $_.CN } | Select-Object -ExpandProperty CN
Without the Select-Object, this will get you an array of objects with three
properties: CN, DistinguishedName, Description.
$ADcomputersInWsus = $GetADComp | Where-Object { $WSUSArr -contains $_.CN }
# An object array like this is perfect for saving as CSV:
# $ADcomputersInWsus | Export-Csv -Path 'X:\ADcomputersInWsus.csv' -NoTypeInformation
2.
Get a list of computers that are in WSUS, but not in AD:
# ($GetADComp).CN returns an string array with just the CN's, just like the $WSUSArr
$WsusComputersNotInAD = $WSUSArr | Where-Object { ($GetADComp).CN -notcontains $_ }
3.
Get a list of AD computers that are not in WSUS:
$ADcomputersNotInWsus = $GetADComp | Where-Object { $WSUSArr -notcontains $_.CN } | Select-Object -ExpandProperty CN
Note, the $WsusComputersNotInAD is derived from the $WSUSArr string array and is therefore also an array of strings, not objects.
To save that to file, either use:
$WsusComputersNotInAD | Out-File -FilePath "C:\XXX\XXX\WSUSCompNotInAD.txt" -Force
Or convert to an object array and use Export-Csv like the other results:
$WsusComputersNotInAD | ForEach-Object { [PsCustomObject]#{'ComputerName' = $_}} |
Export-Csv -Path "C:\XXX\XXX\WSUSCompNotInAD.csv" -NoTypeInformation

Storing and reading objects in to objects or array

I need to create VEEAM Replication jobs. When creating the job I need to provide a list of SourceNetworks and matching TargetNetworks. I have a CSV file that has the matching list in text and then I run a cmdlet to retrieve the matching network object.
CSV:
SourcePortGroup, TargetPortGroup
VLAN 103,LAN0_DMZ
VLAN 120,LAN0_JDE
VLAN 121,LAN0_IT-BDC
I wrote a foreach in which I retrieve the network object using:
foreach ($item in $csvlist) {
Get-VBRServer -Name $SourceESXi | Get-VBRViServerNetworkInfo | Where-Object {
$_.NetworkName -eq $Mapping.SourcePortGroup
}
Get-VBRServer -Name $TargetESXi | Get-VBRViServerNetworkInfo | Where-Object {
$_.NetworkName -eq $Mapping.TargetPortGroup
}
}
This works when debugging, I get the correct result, which is an object. But now I need to store each of them in a new object or in an array, so that later on when creating the job I can easily use the source and target mapping.
I have no clue on what the best way is to store the results and then call them when needed.
Assuming that each Get-VBRServer statement produces only a single item per iteration you could for example use a hashtable for mapping source to target networks:
$networks = #{}
foreach ($item in $csvlist) {
$key = '{0}/{1}' -f
$networks[$key] = #{
'Source' = Get-VBRServer -Name $SourceESXi |
Get-VBRViServerNetworkInfo |
Where-Object { $_.NetworkName -eq $Mapping.SourcePortGroup }
'Target' = Get-VBRServer -Name $TargetESXi |
Get-VBRViServerNetworkInfo |
Where-Object { $_.NetworkName -eq $Mapping.TargetPortGroup }
}
}

Get all unique substrings from array in Powershell

I have a set of strings gathered from logs that I'm trying to parse into unique entries:
function Scan ($path, $logPaths, $pattern)
{
$logPaths | % `
{
$file = $_.FullName
Write-Host "`n[$file]"
Get-Content $file | Select-String -Pattern $pattern -CaseSensitive - AllMatches | % `
{
$regexDateTime = New-Object System.Text.RegularExpressions.Regex "((?:\d{4})-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}(,\d{3})?)"
$matchDate = $regexDateTime.match($_)
if($matchDate.success)
{
$loglinedate = [System.DateTime]::ParseExact($matchDate, "yyyy-MM-dd HH:mm:ss,FFF", [System.Globalization.CultureInfo]::InvariantCulture)
if ($loglinedate -gt $laterThan)
{
$date = $($_.toString().TrimStart() -split ']')[0]
$message = $($_.toString().TrimStart() -split ']')[1]
$messageArr += ,$date,$message
}
}
}
$messageArr | sort $message -Unique | foreach { Write-Host -f Green $date$message}
}
}
So for this input:
2015-09-04 07:50:06 [20] WARN Core.Ports.Services.ReferenceDataCheckers.SharedCheckers.DocumentLibraryMustExistService - A DocumentLibrary 3 could not be found.
2015-09-04 07:50:06 [20] WARN Core.Ports.Services.ReferenceDataCheckers.SharedCheckers.DocumentLibraryMustExistService - A DocumentLibrary 3 could not be found.
2015-09-04 07:50:16 [20] WARN Brighter - The message abc123 has been marked as obsolete by the consumer as the entity has a higher version on the consumer side.
Only the second two entries should be returned
I'm having trouble filtering out duplicates of $message: currently all entries are being returned (sort -Unique is not behaving as I would expect it to). I also need the correct $date to be returned against the filtered $message.
I'm pretty stuck with this, can anyone help?
We can do what you want, but first let's backup just a little bit to help us do this better. Right now you have an array of arrays, and that's difficult to work with in general. What would be better is if you had an array of objects, and those objects had properties such as Date and Message. Let's start there.
if ($loglinedate -gt $laterThan)
{
$date = $($_.toString().TrimStart() -split ']')[0]
$message = $($_.toString().TrimStart() -split ']')[1]
$messageArr += ,$date,$message
}
is going to become...
if ($loglinedate -gt $laterThan)
{
[Array]$messageArr += [PSCustomObject]#{
'date' = $($_.toString().TrimStart() -split ']')[0]
'message' = $($_.toString().TrimStart() -split ']')[1]
}
}
That produces an array of objects, and each object has two properties, Date and Message. That will be much easier to work with.
If you only want the latest version of any message that's easily done with the Group-Object command as such:
$FilteredArr = $messageArr | Group Message | ForEach{$_.Group|sort Date|Select -Last 1}
Then if you want to display it to screen like you are, you could do:
$Filtered|ForEach{Write-Host -f Green ("{0}`t{1}" -f $_.Date, $_.Message)}
My take (not tested) :
function Scan ($path, $logPaths, $pattern)
{
$regex = '(\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2})\s(.+)'
$ht = #{}
$logPaths | % `
{
$file = $_.FullName
Write-Host "`n[$file]"
Get-Content $file | Select-String -Pattern $pattern -CaseSensitive -AllMatches | % `
{
if ($_.line -match $regex -and $ht[$matches[2]] -gt $matches[1])
{ $ht[$matches[2]] = $matches[1] }
}
$ht.GetEnumerator() |
sort Value |
foreach { Write-Host -f Green "$($_.Value)$($_.Name)" }
}
}
This splits the file at the timestamp, and loads the parts into a hash table, using the error message as the key and the timestamp as the data (this will de-dupe the messages in-stream).
The timestamps are already in string-sortable format (yyyy-MM-dd HH:mm:ss), so there's really no need to cast them to [datetime] to find the latest one. Just do a straight string compare, and if the incoming timestamp is greater than an existing value for that message, replace the existing value with the new one.
When you're done, you should have a hash table with a key for each unique message found, having a value of the latest timestamp found for that message.

How to remove item from an array in PowerShell?

I'm using Powershell 1.0 to remove an item from an Array. Here's my script:
param (
[string]$backupDir = $(throw "Please supply the directory to housekeep"),
[int]$maxAge = 30,
[switch]$NoRecurse,
[switch]$KeepDirectories
)
$days = $maxAge * -1
# do not delete directories with these values in the path
$exclusionList = Get-Content HousekeepBackupsExclusions.txt
if ($NoRecurse)
{
$filesToDelete = Get-ChildItem $backupDir | where-object {$_.PsIsContainer -ne $true -and $_.LastWriteTime -lt $(Get-Date).AddDays($days)}
}
else
{
$filesToDelete = Get-ChildItem $backupDir -Recurse | where-object {$_.PsIsContainer -ne $true -and $_.LastWriteTime -lt $(Get-Date).AddDays($days)}
}
foreach ($file in $filesToDelete)
{
# remove the file from the deleted list if it's an exclusion
foreach ($exclusion in $exclusionList)
{
"Testing to see if $exclusion is in " + $file.FullName
if ($file.FullName.Contains($exclusion)) {$filesToDelete.Remove($file); "FOUND ONE!"}
}
}
I realize that Get-ChildItem in powershell returns a System.Array type. I therefore get this error when trying to use the Remove method:
Method invocation failed because [System.Object[]] doesn't contain a method named 'Remove'.
What I'd like to do is convert $filesToDelete to an ArrayList and then remove items using ArrayList.Remove. Is this a good idea or should I directly manipulate $filesToDelete as a System.Array in some way?
Thanks
The best way to do this is to use Where-Object to perform the filtering and use the returned array.
You can also use #splat to pass multiple parameters to a command (new in V2). If you cannot upgrade (and you should if at all possible, then just collect the output from Get-ChildItems (only repeating that one CmdLet) and do all the filtering in common code).
The working part of your script becomes:
$moreArgs = #{}
if (-not $NoRecurse) {
$moreArgs["Recurse"] = $true
}
$filesToDelete = Get-ChildItem $BackupDir #moreArgs |
where-object {-not $_.PsIsContainer -and
$_.LastWriteTime -lt $(Get-Date).AddDays($days) -and
-not $_.FullName.Contains($exclusion)}
In PSH arrays are immutable, you cannot modify them, but it very easy to create a new one (operators like += on arrays actually create a new array and return that).
I agree with Richard, that Where-Object should be used here. However, it's harder to read.
What I would propose:
# get $filesToDelete and #exclusionList. In V2 use splatting as proposed by Richard.
$res = $filesToDelete | % {
$file = $_
$isExcluded = ($exclusionList | % { $file.FullName.Contains($_) } )
if (!$isExcluded) {
$file
}
}
#the files are in $res
Also note that generally it is not possible to iterate over a collection and change it. You would get an exception.
$a = New-Object System.Collections.ArrayList
$a.AddRange((1,2,3))
foreach($item in $a) { $a.Add($item*$item) }
An error occurred while enumerating through a collection:
At line:1 char:8
+ foreach <<<< ($item in $a) { $a.Add($item*$item) }
+ CategoryInfo : InvalidOperation: (System.Collecti...numeratorSimple:ArrayListEnumeratorSimple) [], RuntimeException
+ FullyQualifiedErrorId : BadEnumeration
This is ancient. But, I wrote these a while ago to add and remove from powershell lists using recursion. It leverages the ability of powershell to do multiple assignment . That is, you can do $a,$b,$c=#('a','b','c') to assign a b and c to their variables. Doing $a,$b=#('a','b','c') assigns 'a' to $a and #('b','c') to $b.
First is by item value. It'll remove the first occurrence.
function Remove-ItemFromList ($Item,[array]$List(throw"the item $item was not in the list"),[array]$chckd_list=#())
{
if ($list.length -lt 1 ) { throw "the item $item was not in the list" }
$check_item,$temp_list=$list
if ($check_item -eq $item )
{
$chckd_list+=$temp_list
return $chckd_list
}
else
{
$chckd_list+=$check_item
return (Remove-ItemFromList -item $item -chckd_list $chckd_list -list $temp_list )
}
}
This one removes by index. You can probably mess it up good by passing a value to count in the initial call.
function Remove-IndexFromList ([int]$Index,[array]$List,[array]$chckd_list=#(),[int]$count=0)
{
if (($list.length+$count-1) -lt $index )
{ throw "the index is out of range" }
$check_item,$temp_list=$list
if ($count -eq $index)
{
$chckd_list+=$temp_list
return $chckd_list
}
else
{
$chckd_list+=$check_item
return (Remove-IndexFromList -count ($count + 1) -index $index -chckd_list $chckd_list -list $temp_list )
}
}
This is a very old question, but the problem is still valid, but none of the answers fit my scenario, so I will suggest another solution.
I my case, I read in an xml configuration file and I want to remove an element from an array.
[xml]$content = get-content $file
$element = $content.PathToArray | Where-Object {$_.name -eq "ElementToRemove" }
$element.ParentNode.RemoveChild($element)
This is very simple and gets the job done.

Resources