I'm currently having a bit of trouble with my current project. I have two arrays - the first array contains reference values for disk size:
$RequiredDisks0 = New-Object System.Object
$RequiredDisks0 | Add-Member -Type NoteProperty -Name "DeviceID" -Value "C:"
$RequiredDisks0 | Add-Member -Type NoteProperty -Name "SizeGB" -Value "80"
The second array contains the disk information of the underlying system:
$SystemDisks = Get-WmiObject Win32_LogicalDisk |
Where {$_.DriveType -eq 3} |
select DeviceID,
#{Name="Size(GB)";Expression={[decimal]("{0:N0}" -f ($_.Size/1gb))}}
What I would like to do, is check the given array against the reference array to see if any given disks are smaller than required. I've found out that I can compare the arrays by using
Compare-Object -ReferenceObject $RequiredDisks -DifferenceObject $SystemDisks -Property SizeGB,DeviceID
And I indeed receive the differences as follows:
SizeGB DeviceID SideIndicator
------ -------- -------------
99 C: =>
15 H: =>
100 I: =>
80 C: <=
25 H: <=
200 I: <=
Where I'm having trouble is working with the output. The result I'd like to achieve is an output stating "Disk n is smaller than required!". I know that everything with the side indicator "<=" is the required value and everything with the "=>" side indicator is the given value. I've tried a foreach statement but I am unable to process the data as needed - I need to check the given value against the required value and if it's smaller, tell me so. How can I again compare these values as required? Basically a "foreach object where SideIndicator is <= compare to object where SideIndicator is => and DeviceID equals DeviceID". How do I translate that into proper code?
It looks to me like the Compare-Object is doing a double comparison on both properties. The documentation or another StackOverflow soul may be able to help with that command.
My approach would be to translate your pseudo-code into code:
foreach ($disk in $SystemDisks){
$ref = $RequiredDisks | Where-object {$_.DeviceID -eq $disk.DeviceID}
if([int]($disk.SizeGB) -lt [int]($ref.SizeGB){
Write-Output "Disk $($disk.DeviceID) is smaller than required!"
}
}
Related
I have a rather peculiar nested JSON where in some instances a key - value pair occurs as normal, but in others the type of the key appears in a further nesting.
{"metadata":{"systemId":"da1895","legalEntity":"A0"},"recordContent":{"positionDate":"2019-04-08 00:00:00.0","account":{"string":"G32"},"seg":{"string":"S"},"strike":{"double":4.4}}}
{"metadata":{"systemId":"45364d","legalEntity":"5G"},"recordContent":{"positionDate":"2019-04-08 00:00:00.0","account":{"string":"G81"},"seg":{"string":"S"},"strike":{"double":5.0}}}
In the example you can see metadata's fields are straightforward key-value pairs, but underneath recordContent, we have positionDate which is a straightforward key-value but "account":{"string":"G32"} and "strike":{"double":4.4} are not.
I'd like to ditch the type information and arrive at a CSV structure as follows:
systemId, legalEntity, positionDate, account,seg,strike
da1895, A0, 2019-04-08 00:00:00.0,G32, S, 4.4
4536d, 5G, 2019-04-08 00:00:00.0,G81, S, 5.0
Any ideas on how to convert such a structure to CSV using Powershell?
Here's what I tried:
$TemplateParametersFile = "c:\data\output.json"
$JsonParameters = Get-Content $TemplateParametersFile | ConvertFrom-Json
$metadatafields = $JsonParameters.metadata[0].PSObject.Properties.Name
$recordcontentfields = $JsonParameters.recordContent[0].PsObject.Properties.Name
$oData = New-Object PSObject
$metadatafields |
ForEach {
Add-Member -InputObject $oData -NotePropertyName ($_) -NotePropertyValue $JsonParameters.metadata.($_)
}
$recordcontentfields |
ForEach {
Add-Member -InputObject $oData -NotePropertyName ($_) -NotePropertyValue $JsonParameters.recordContent.($_)
}
This gave me:
$oData
systemId : {da1895, 45364d}
legalEntity : {A0, 5G}
positionDate : {2019-04-08 00:00:00.0, 2019-04-08 00:00:00.0}
account : {#{string=G32}, #{string=G81}}
seg : {#{string=S}, #{string=S}}
strike : {#{double=4.4}, #{double=5.0}}
I'm a bit stuck now and the above doesn't convert to csv.
Note that other than metadata and recordContent, I've not hardcoded any fieldnames and I'd like to maintain that flexibility in case the JSON structure changes.
Thanks
I suggest collecting the property-name-value pairs iteratively in an ordered hashtable ([ordered] #{}), which can then be cast to [pscustomobject] to convert it to a custom object.
No property names are hard-coded in the following solution, but the object-graph structure is assumed to follow the pattern in your sample JSON, which is limited to one level of nesting - if you need to process arbitrarily nested objects, this answer may be a starting point.
Reflection (discovery of the property names and values) is performed via the intrinsic .psobject property that PowerShell makes available on all objects.
# Parse sample JSON into an array of [pscustomobject] graphs.
$fromJson = ConvertFrom-Json #'
[
{"metadata":{"systemId":"da1895","legalEntity":"A0"},"recordContent":{"positionDate":"2019-04-08 00:00:00.0","account":{"string":"G32"},"seg":{"string":"S"},"strike":{"double":4.4}}}
,
{"metadata":{"systemId":"45364d","legalEntity":"5G"},"recordContent":{"positionDate":"2019-04-08 00:00:00.0","account":{"string":"G81"},"seg":{"string":"S"},"strike":{"double":5.0}}}
]
'#
# Initialize an aux. ordered hashtable to collect the property-name-value
# pairs in.
$oht = [ordered] #{}
$fromJson | ForEach-Object {
$oht.Clear()
# Loop over top-level properties.
foreach ($topLevelProp in $_.psobject.Properties) {
# Loop over second-level properties.
foreach ($prop in $topLevelProp.Value.psobject.Properties) {
if ($prop.Value -is [System.Management.Automation.PSCustomObject]) {
# A nested value: Use the value of the (presumed to be one-and-only)
# property of the object stored in the value.
$oht[$prop.Name] = $prop.Value.psobject.Properties.Value
}
else {
# A non-nested value: use as-is.
$oht[$prop.Name] = $prop.Value
}
}
}
# Construct and output a [pscustomobject] from the aux. ordered hashtble.
[pscustomobject] $oht
} |
ConvertTo-Csv # Replace this with Export-Csv to export to a file.
The above yields:
"systemId","legalEntity","positionDate","account","seg","strike"
"da1895","A0","2019-04-08 00:00:00.0","G32","S","4.4"
"45364d","5G","2019-04-08 00:00:00.0","G81","S","5"
A few years ago, I wrote a reusable Flatten-Object function for this.
The only difference is that it combines the (sub)property names with the parent property names as they might not be unique:
$JsonParameters |Flatten-Object |Format-Table
metadata.systemId metadata.legalEntity recordContent.positionDate recordContent.account.string recordContent.seg.string recordContent.strike.double
----------------- -------------------- -------------------------- ---------------------------- ------------------------ ---------------------------
da1895 A0 2019-04-08 00:00:00.0 G32 S 4.4
45364d 5G 2019-04-08 00:00:00.0 G81 S 5
Try this:
$data = ConvertFrom-Json #"
[
{"metadata":{"systemId":"da1895","legalEntity":"A0"},"recordContent":{"positionDate":"2019-04-08 00:00:00.0","account":{"string":"G32"},"seg":{"string":"S"},"strike":{"double":4.4}}},
{"metadata":{"systemId":"45364d","legalEntity":"5G"},"recordContent":{"positionDate":"2019-04-08 00:00:00.0","account":{"string":"G81"},"seg":{"string":"S"},"strike":{"double":5.0}}}
]
"#
$data | Select-Object -Property #{l="systemId"; e={$_.metadata.systemId}}, #{l="legalEntity"; e={$_.metadata.legalEntity}},
#{l="positionDate"; e={$_.recordContent.positionDate}}, #{l="account"; e={$_.recordContent.account.string}},
#{l="seg"; e={$_.recordContent.seg.string}}, #{l="strike"; e={$_.recordContent.strike.double}} | Export-Csv
This should work with any nested psobject.
$json = #'
{"metadata":{"systemId":"da1895","legalEntity":"A0"},"recordContent":{"positionDate":"2019-04-08 00:00:00.0","account":{"string":"G32"},"seg":{"string":"S"},"strike":{"double":4.4}}}
'#
$obj = ConvertFrom-Json $json
$obj.recordContent | gm -MemberType NoteProperty | % {
$prop = $_.name
if ($obj.recordContent.$prop.GetType().name -eq 'pscustomobject') {
$obj.recordContent.$prop = $obj.recordContent.$prop.psobject.Members | where membertype -eq noteproperty | select -ExpandProperty value
}
$obj.metadata | add-member -MemberType NoteProperty -Name $prop -Value $obj.recordContent.$prop
}
$newobj = $obj.metadata
$newobj
I'm trying to find the row with an attribute that is larger than the other row's attributes. Example:
$Array
Name Value
---- ----
test1 105
test2 101
test3 512 <--- Selects this row as it is the largest value
Here is my attempt to '1 line' this but It doesn't work.
$Array | % { If($_.value -gt $Array[0..($Array.Count)].value){write-host "$_.name is the largest row"}}
Currently it outputs nothing.
Desired Output:
"test1 is the largest row"
I'm having trouble visualizing how to do this efficiently with out some serious spaghetti code.
You could take advantage of Sort-Object to rank them by the property "Value" like this
$array = #(
[PSCustomObject]#{Name='test1';Value=105}
[PSCustomObject]#{Name='test2';Value=101}
[PSCustomObject]#{Name='test3';Value=512}
)
$array | Sort-Object -Property value -Descending | Select-Object -First 1
Output
Name Value
---- -----
test3 512
To incorporate your write host you can just run the one you select through a foreach.
$array | Sort-Object -Property value -Descending |
Select-Object -First 1 | Foreach-Object {Write-host $_.name,"has the highest value"}
test3 has the highest value
Or capture to a variable
$Largest = $array | Sort-Object -Property value -Descending | Select-Object -First 1
Write-host $Largest.name,"has the highest value"
test3 has the highest value
PowerShell has many built in features to make tasks like this easier.
If this is really an array of PSCustomObjects you can do something like:
$Array =
#(
[PSCustomObject]#{ Name = 'test1'; Value = 105 }
[PSCustomObject]#{ Name = 'test2'; Value = 101 }
[PSCustomObject]#{ Name = 'test3'; Value = 512 }
)
$Largest = ($Array | Sort-Object Value)[-1].Name
Write-host $Largest,"has the highest value"
This will sort your array according to the Value property. Then reference the last element using the [-1] syntax, then return the name property of that object.
Or if you're a purist you can assign the variable like:
$Largest = $Array | Sort-Object Value | Select-Object -Last 1 -ExpandProperty Name
If you want the whole object just remove .Name & -ExpandProperty Name respectively.
Update:
As noted PowerShell has some great tools to help with common tasks like sorting & selecting data. However, that doesn't mean there's never a need for looping constructs. So, I wanted to make a couple of points about the OP's own answer.
First, if you do need to reference array elements by index use a traditional For loop, which might look something like:
For( $i = 0; $i -lt $Array.Count; ++$i )
{
If( $array[$i].Value -gt $LargestValue )
{
$LargestName = $array[$i].Name
$LargestValue = $array[$i].Value
}
}
$i is commonly used as an iteration variable, and within the script block is used as the array index.
Second, even the traditional loop is unnecessary in this case. You can stick with the ForEach loop and track the largest value as and when it's encountered. That might look something like:
ForEach( $Row in $array )
{
If( $Row.Value -gt $LargestValue )
{
$LargestName = $Row.Name
$LargestValue = $Row.Value
}
}
Strictly speaking you don't need to assign the variables beforehand, though it may be a good practice to precede either of these with:
$LargestName = ""
$LargestValue = 0
In these examples you'd have to follow with a slightly modified Write-Host command
Write-host $LargestName,"has the highest value"
Note: Borrowed some of the test code from Doug Maurer's Fine Answer. Considering our answers were similar, this was just to make my examples more clear to the question and easier to test.
Figured it out, hopefully this isn't awful:
$Count = 1
$CurrentLargest = 0
Foreach($Row in $Array) {
# Compare This iteration vs the next to find the largest
If($Row.value -gt $Array.Value[$Count]){$CurrentLargest = $Row}
Else {$CurrentLargest = $Array[$Count]}
# Replace the existing largest value with the new one if it is larger than it.
If($CurrentLargest.Value -gt $Largest.Value){ $Largest = $CurrentLargest }
$Count += 1
}
Write-host $Largest.name,"has the highest value"
Edit: its awful, look at the other answers for a better way.
I'm quite new in PowerShell and I would need a little bit of support how to replace values in an array. Please have a look at my example:
[array[]]$nodes = #()
[array[]]$nodes = get-NcNode | select-object -property Node, #{Label = "slot"; expression = {#("a")*4}}
$nodes
Node slot
---- ----
nn01 {a,a,a,a}
nn02 {a,a,a,a}
nn03 {a,a,a,a}
nn04 {a,a,a,a}
$nodes[0].slot[0]
a
$nodes[0].slot[0] = "b" #I try to replace a with b
$nodes[0].slot[0]
a #It didn’t work
$nodes[0].slot.SetValue("b",0) #I try to replace a with b
$nodes[0].slot[0]
a #It didn’t work
$nodes[0] | Add-Member -MemberType NoteProperty -Name slot[0] -Value "b" -Force
$nodes[0]
Node slot slot[0]
---- ---- -------
nn01 {a,a,a,a} b #That’s not what I wanted
If you really need an array of arrays (type [array[]]), your problem is solved as follows:
$nodes[0][0].slot[0] = "b"
That is, each of your $nodes elements is itself an array, and the way you filled $nodes, each [pscustomobject] instance output by your get-NcNode | select-object ... pipeline became its own element of $nodes, but each as a single-element sub-array - hence the need for the extra [0] index access.[1]
However, it sounds like a regular array ([array], effectively the same as [object[]]) is sufficient in your case, where each element holds a (single, scalar) [pscustomobject]:
# Type constraint [array] creates a regular [object[]] array.
[array] $nodes = get-NcNode | select-object -property Node, #{Label = "slot"; expression = {#("a")*4}}
With $nodes defined like this, your original code should work.
[1] On getting a value - but not on setting - you can get away without the extra index, thanks to PowerShell's member-access enumeration feature.
I have a large set of data roughly 10 million items that I need to process efficiently and quickly removing duplicate items based on two of the six column headers.
I have tried grouping and sorting items but it's horrendously slow.
$p1 = $test | Group-Object -Property ComputerSeriaID,ComputerID
$p2 = foreach ($object in $p1.group) {
$object | Sort-Object -Property FirstObserved | Select-Object -First 1
}
The goal would be to remove duplicates by assessing two columns while maintaining the oldest record based on first observed.
The data looks something like this:
LastObserved : 2019-06-05T15:40:37
FirstObserved : 2019-06-03T20:29:01
ComputerName : 1
ComputerID : 2
Virtual : 3
ComputerSerialID : 4
LastObserved : 2019-06-05T15:40:37
FirstObserved : 2019-06-03T20:29:01
ComputerName : 5
ComputerID : 6
Virtual : 7
ComputerSerialID : 8
LastObserved : 2019-06-05T15:40:37
FirstObserved : 2019-06-03T20:29:01
ComputerName : 9
ComputerID : 10
Virtual : 11
ComputerSerialID : 12
You might want to clean up your question a little bit, because it's a little bit hard to read, but I'll try to answer the best I can with what I can understand about what you're trying to do.
Unfortunately, with so much data there's no way to do this quickly. String Comparison and sorting are done by brute force; there is no way to reduce the complexity of comparing each character in one string against another any further than measuring them one at a time to see if they're the same.
(Honestly, if this were me, I'd just use export-csv $object and perform this operation in excel. The time tradeoff to scripting something like this only once just wouldn't be worth it.)
By "Items" I'm going to assume that you mean rows in your table, and that you're not trying to retrieve only the strings in the rows you're looking for. You've already got the basic idea of select-object down, you can do that for the whole table:
$outputFirstObserved = $inputData | Sort-Object -Property FirstObserved -Unique
$outputLastObserved = $inputData | Sort-Object -Property LastObserved -Unique
Now you have ~20 million rows in memory, but I guess that beats doing it by hand. All that's left is to join the two tables. You can download that Join-Object command from the powershell gallery with Install-Script -Name Join and use it in the way described. If you want to do this step yourself, the easiest way would be to squish the two tables together and sort them again:
$output = $outputFirstObserved + $outputLastObserved
$return = $output | Sort-Object | Get-Unique
Does this do it? It keeps the one it finds first.
$test | sort -u ComputerSeriaID, ComputerID
I created this function to de-duplicate my multi-dimensional arrays.
Basically, I concatenate the contents of the record, add this to a hash.
If the concatenate text already exists in the hash, don't add it to the array to be returned.
Function DeDupe_Array
{
param
(
$Data
)
$Return_Array = #()
$Check_Hash = #{}
Foreach($Line in $Data)
{
$Concatenated = ''
$Elements = ($Line | Get-Member -MemberType NoteProperty | % {"$($_.Name)"})
foreach($Element in $Elements)
{
$Concatenated += $line.$Element
}
If($Check_Hash.$Concatenated -ne 1)
{
$Check_Hash.add($Concatenated,1)
$Return_Array += $Line
}
}
return $Return_Array
}
Try the following script.
Should be as fast as possible due to avoiding any pipe'ing in PS.
$hashT = #{}
foreach ($item in $csvData) {
# Building hash table key
$key = '{0}###{1}' -f $item.ComputerSeriaID, $item.ComputerID
# if $key doesn't exist yet OR when $key exists and "FirstObserverd" is less than existing one in $hashT (only valid when date provided in sortable format / international format)
if ((! $hashT.ContainsKey($key)) -or ( $item.FirstObserved -lt $hashT[$key].FirstObserved )) {
$hashT[$key] = $item
}
}
$result = $hashT.Values
The purpose of my script is to get the entire list of VMs from our vcenter, check to see if they have DNS entries - if the dns entry is null then output the VM name and corresponding IP address. Pretty simple, but I'm getting tripped up combining the VM name array with the IP address array. Here's what I mean:
$mainArray = #()
$vmArray = #()
$ipArray = #()
$vms = Get-VM | select Name, #{N="DnsName"; E={$_.ExtensionData.Guest.Hostname}}, #{N="IPAddress"; E={$_.Guest.IPAddress[0]}}
foreach ($vm in $vms)
{
$dnsname = $vm.DnsName
$ipaddr = $vm.IPAddress
$vmname = $vm.Name
if (!$vm.DNSName) #if DNS name is null
{
$vmArray += $vmname
$ipArray += $ipaddr
}
}
$item = New-Object PSObject
$item | Add-Member -type NoteProperty -Name 'VM Name' -Value $vmArray
$item | Add-Member -type NoteProperty -Name 'IP Address' -Value $ipArray
$mainArray += $item
$mainArray | ft -wrap -autosize
This outputs the two columns next to each other, but the output is separated by commas and is truncated with an ellipse. I know that there's a better way to do this, so I'm open to suggestions. Also be curious to know how to make this work just for my own understanding. Appreciate the help everyone.
Your main issue is that you are populating an array outside your loop. I don't have access to PowerCLI right in front of me now but we should be able to remove most of that code as it appears to be redundant and needless. Continuing after $vms being declared
$vms | Where-Object{!$_.DNSName} | ForEach-Object{
New-Object PSObject -Property #{
'VM Name' = $_.Name
'IP Address' = $_.IPAddress
}
} | ft -wrap -autosize
Before you made a two arrays. One for names and the other ips. Then you took those 2 arrays and made a single object with those two properties. I removed the need for the if statement with the Where-Object{!$_.DNSName}. Now all the objects being processed are the ones you want.
You could really do this is in one/two lines using the same logic.
Get-VM | Select #{N='VM Name';E={$_.Name}}, #{N="DnsName"; E={$_.ExtensionData.Guest.Hostname}}, #{N="IP Address"; E={$_.Guest.IPAddress[0]}} |
Where-Object{!$_.DNSName} | Select "VM Name","IP Address" | Format-Table -wrap -autosize
Again, I cannot test but it should work. If there are any errors they should be easily correctable. Assuming I figured out your issue in the first place.
About the ellipses
PowerShell has a variable $FormatEnumerationLimit that governs how many array elements are display in console output. There is a good article that discusses it on TechNet
By default $FormatEnumerationLimit is set to 4
Matt, I had found that link earlier but didn't find it very helpful since it makes the output garbled and doesn't breakout the values by row. This is how I was able to make the original script work:
$i=0
write-output $vmArray |
foreach {
new-object psobject -property #{
VMName = $_
IPAddress = $ipArray[$i++]
}
} | ft -auto -wrap
Rather unconventional, but does work nonetheless. There really should be an easier way to combine array side by side, but I digress. Thanks a ton Matt.
Update: This is probably more conventional:
For($i=0;$i -lt $vmArray.Count;$i++)
{
$item = New-Object PSObject
$item | Add-Member -type NoteProperty -Name 'VM Name' -Value $vmArray[$i]
$item | Add-Member -type NoteProperty -Name 'IP Address' -Value $ipArray[$i]
$mainArray += $item
}
$mainArray | ft -wrap -autosize