OverloadDefinitions on CSV field? - arrays

I created a question earlier where I asked how to save a specific CSV column into a PowerShell array. I got my question answered and my passwords and my usernames are now saved in two different arrays. However, the Server IP-Adresses doesn't want to be saved into the array I made for them to be saved in. When I try to log in to a server (with PowerShell inputting credentials automatically) I get this error:
Invoke-Command : One or more computer names are not valid. If you are trying to
pass a URI, use the -ConnectionUri parameter, or pass URI objects instead of
strings.
At C:xxx\xxx\xxx\xxx\pornfolder:37 char:15
+ ... $output = Invoke-Command -computername $AddressArray[$row] -credent ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidArgument: (System.String[]:String[]) [Invoke-Command], ArgumentException
+ FullyQualifiedErrorId : PSSessionInvalidComputerName,Microsoft.PowerShell.Commands.InvokeCommandCommand
So I printed the array where the adresses are meant to be saved in, and I get this:
OverloadDefinitions
-------------------
System.Object&, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089 Address(int )
This is, honest to god, all that is saved in the array. No wonder the computername(s) are invalid.
Here is the code
$PasswordsArray = New-Object System.Collections.ArrayList
$UsernamesArray = New-Object System.Collections.ArrayList
$Importedcsv = Import-csv "C:\my\csv\file\is\located.here" -Delimiter ";"
$PasswordsArray += #($Importedcsv.password)
$AA += [string] #($Importedcsv.address)
$UsernamesArray += #($Importedcsv.username)

Basically your issue is caused by a convenience feature that was introduced in PowerShell v3 combined with awkward field naming. Since PowerShell v3 arrays are automatically unrolled when you use dot-notation on them ($arr.something). That way properties or methods can be called on the array elements via the array object without having to loop over the elements ($arr | ForEach-Object {$_.something}).
Your column title (address) conflicts with the name of a method of the array object (Address()).
PS C:\> $Importedcsv | Get-Member
TypeName: System.Management.Automation.PSCustomObject
Name MemberType Definition
---- ---------- ----------
Equals Method bool Equals(System.Object obj)
GetHashCode Method int GetHashCode()
GetType Method type GetType()
ToString Method string ToString()
address NoteProperty System.String address=192.168.0.10
password NoteProperty System.String password=#adfgad
username NoteProperty System.String username=advokathuset\user
PS C:\> Get-Member -InputObject $Importedcsv
TypeName: System.Object[]
Name MemberType Definition
---- ---------- ----------
Count AliasProperty Count = Length
Add Method int IList.Add(System.Object value)
Address Method System.Object&, mscorlib, Versio...
Clear Method void IList.Clear()
...
$Importedcsv | Get-Member unrolls the array (via the pipeline), so Get-Member is called on the array elements, whereas Get-Member -InputObject $Importedcsv operates on the array object itself (PowerShell imports CSVs as an array of objects).
Rename the column to something else, either at the source (where you create the file), or on import:
$filename = "C:\my\csv\file\is\located.here"
$headers = 'IPAddress', 'Username', 'Password'
$Importedcsv = Import-Csv $filename -Delimiter ";" -Header $headers
$AA = #($Importedcsv.IPAddress)
or expand the address property on the individual array items in a loop:
$AA = #($Importedcsv | ForEach-Object { $_.address } )
Note that a construct [string]#(...) will cast your array to a single string, so don't use it unless you actually want all addresses in a single string. To enforce a string array you'd use [string[]]#(...), but usually that's not necessary with PowerShell, so I'd recommend against using it unless you know what you need it for.
Also note that it's unnecessary to create an ArrayList object and append to it. The #() operator will already produce an array, so you should simplify things like this:
$PasswordsArray = New-Object System.Collections.ArrayList
$PasswordsArray += #($Importedcsv.password)
to this:
$PasswordsArray = #($Importedcsv.password)
unless you require specific features of an ArrayList.

Related

Powershell Compare 2 Arrays of Hashtables based on a property value

I have one array of hashtables like the one below:
$hashtable1 = #{}
$hashtable1.name = "aaa"
$hashtable1.surname =#()
$hashtable1.surname += "bbb"
$hashtable2 = #{}
$hashtable2.name = "aaa"
$hashtable2.surname =#()
$hashtable2.surname += "ccc"
$hashtable3 = #{}
$hashtable3.name = "bbb"
$hashtable3.surname = #()
$hashtable3.surname += "xxx"
$A = #($hashtable1; $hashtable2; $hashtable3)
I need to iterate though the array and I need to find out duplicates based on hashtable[].name
Then I need to group those hashtable.surname to hashtable[].surname so that the result will be an array of hashtables that will group all for name all the surnames:
$hashtable1.name = "aaa"
$hashtable1.surname = ("bbb","ccc")
$hashtable3.name = "bbb"
$hashtable3.surname = ("xxx")
I was looking into iterating to empty array
+
I have found this link:
powershell compare 2 arrays output if match
but I am not sure on how to reach into the elements of the hashtable.
My options:
I was wondering if -contain can do it.
I have read about compare-object but I am not sure it can be done like that.
(It looks a bit scary in the moment)
I am on PS5.
Thanks for your help,
Aster
You can group your array items by the names using a scriptblock like so.
Once grouped, you can easily build your output to do what you seek.
#In PS 7.0+ you can use Name directly but earlier version requires the use of the scriptblock when dealing with arrays of hashtables.
$Output = $A | Group-Object -Property {$_.Name} | % {
[PSCustomObject]#{
Name = $_.Name
Surname = $_.Group.Surname | Sort-Object -Unique
}
}
Here is the output variable content.
Name Surname
---- -------
aaa {bbb, ccc}
bbb xxx
Note
Improvements have been made in PS 7.0 that allows you to use simply the property name (eg: Name) in Group-Object for arrays of hashtables, just like you would do for any other arrays type. For earlier version though, these particular arrays must be accessed by passing the property in a scriptblock, like so: {$_.Name}
References
MSDN - Group_Object
SS64 - Group Object
Dr Scripto - Use a Script block to create custom groupings in PowerShell

How to select object type when exporting to csv in powershell

My array $list contains multiple object types. One is a string and one is a
PSCustomObject.
When I pipe it to Export-Csv, all I get is the first object that is string which returns useless results.
PS C:\data\functions> $list | gm
TypeName: System.String
Name MemberType Definition
---- ---------- ----------
Clone Method System.Object Clone(), System.Object ICloneable.Clone()
CompareTo Method int CompareTo(System.Object value), int CompareTo(string strB), int IComparable.Co...
--- Output removed ----
TypeName: System.Management.Automation.PSCustomObject
Name MemberType Definition
---- ---------- ----------
Equals Method bool Equals(System.Object obj)
GetHashCode Method int GetHashCode()
GetType Method type GetType()
ToString Method string ToString()
Direction NoteProperty string Direction=both
Hostname NoteProperty string Hostname=Desktop1
Target NoteProperty string Target=Rental
PS C:\data\functions> $list| Export-Csv -Path "c:\data\list.txt" -NoTypeInformation
PS C:\data\functions> Get-Content "C:\data\list.txt"
"Length"
"5"
I have tried to cast the object to PSCustomObject by using
[PSCustomObject] $list and [PSCustomObject[]] $list .
I have tried $list | out-File -FilePath "c:\data\list.txt". That does give me the contents of the array as it is displayed in the terminal
hello
Hostname : Desktop1
Target : Rental
Hostname : Desktop2
Target : Rental
Hostname : Desktop3
Target : Rental
Hostname : Desktop4
Target : Rental
I expect to get a CSV-file
I would say you have to select the properties you want, or else it won't save them all.
$a = [pscustomobject]#{name='Joe'}
$b = [pscustomobject]#{address='here'}
$a,$b | select name,address | export-csv all.csv

Powershell script for hash

I have a CSV file (Comma Separated Values)
The file looks like this:
20171108,120909470,SO1244,12,101
20171109,122715740,AG415757,11,101
I need to obscure the data in (for example) columns 3 and, without affecting any of the other entries in the file.
I want to do this using a hashing algorithm like SHA1 or MD5, so that the same strings will resove to the same hash values anywhere they are encountered.
I need to send data to a third party, and certain columns contain sensitive information (e.g. customer names). I need the file to be complete, and where a string is replaced, I need it to be done in the same way every time it is encountered (so that any mapping or grouping remains). It does not need military encryption, just to be difficult to reverse. As I need to to this intermittently, a scripted solution would be ideal.
What is the easiest way to achieve this using a command line tool or script?
By preference, I would like a PowerShell script, since that does not require any additional software to achieve...
This question seems like a duplicate of I need to hash (obfuscate) a column of data in a CSV file. Script preferred but the proposed solution didn't resolve my problem and throws the following error
You cannot call a method on a null-valued expression.
At C:\Users\mey\Hashr.ps1:4 char:5
+ $_.column3 = $_.column3.gethashcode()
The script is the following
(Import-Csv .\results.csv -delimiter ',' ) | ForEach-Object{
$_.column3 = $_.column3.gethashcode()
$_
} | Export-Csv .\myobfuscated.csv -NoTypeInformation -delimiter ','
Update:
Here's the program i am running and that has been proposed by #BaconBits:
param (
[Parameter(Mandatory = $true, ValueFromPipeline = $true, Position = 0)]
[String[]]
$String,
[Parameter(Position = 1)]
[ValidateSet('SHA1', 'MD5', 'SHA256', 'SHA384', 'SHA512')]
[String]
$HashName = 'SHA256'
)
process {
$StringBuilder = [System.Text.StringBuilder]::new(128)
[System.Security.Cryptography.HashAlgorithm]::Create($HashName).ComputeHash([System.Text.Encoding]::UTF8.GetBytes($String)) | ForEach-Object {
[Void]$StringBuilder.Append($_.ToString("x2"))
}
$StringBuilder.ToString()
}
}
$csv = Import-Csv .\results.csv -delimiter ','
foreach ($line in $csv) {
$line.column1 = Get-StringHash $line.column1
}
$csv | Export-Csv .\myobfuscated.csv -NoTypeInformation -delimiter ','
The csv file i am importing is an output from another java program i made and it creates no header, it just fill the csv file with values
I am getting this error
Get-StringHash : Cannot bind argument to parameter 'String' because it is null.
Based on the doc, you're not going to want to use GetHashCode() this way:
A hash code is intended for efficient insertion and lookup in
collections that are based on a hash table. A hash code is not a
permanent value. For this reason:
Do not serialize hash code values or store them in databases.
Do not use the hash code as the key to retrieve an object from a keyed collection.
Do not send hash codes across application domains or processes. In some cases, hash codes may be computed on a per-process or
per-application domain basis.
Do not use the hash code instead of a value returned by a cryptographic hashing function if you need a cryptographically strong
hash. For cryptographic hashes, use a class derived from the
System.Security.Cryptography.HashAlgorithm or
System.Security.Cryptography.KeyedHashAlgorithm class.
Do not test for equality of hash codes to determine whether two objects are equal. (Unequal objects can have identical hash codes.) To
test for equality, call the ReferenceEquals or Equals method.
Bullet point 4 is the main problem. There's no guarantee that the hashing isn't reversible. The hashing function used is an implementation detail, not a secure cryptographic function like SHA.
I'd use a function like this one:
function Get-StringHash {
[CmdletBinding()]
param (
[Parameter(Mandatory = $true, ValueFromPipeline = $true, Position = 0)]
[String[]]
$String,
[Parameter(Position = 1)]
[ValidateSet('SHA1', 'MD5', 'SHA256', 'SHA384', 'SHA512')]
[String]
$HashName = 'SHA256'
)
process {
$StringBuilder = [System.Text.StringBuilder]::new(128)
[System.Security.Cryptography.HashAlgorithm]::Create($HashName).ComputeHash([System.Text.Encoding]::UTF8.GetBytes($String)) | ForEach-Object {
[Void]$StringBuilder.Append($_.ToString("x2"))
}
$StringBuilder.ToString()
}
}
$csv = Import-Csv .\results.csv -delimiter ',' -Header column1,column2,column3,column4,column5
foreach ($line in $csv) {
$line.column3 = Get-StringHash $line.column3
}
$csv | Export-Csv .\myobfuscated.csv -NoTypeInformation -delimiter ','
I believe I based that function off of this one, but it's been awhile since I've written it.
Edit by LotPings to show results of hash
"column1","column2","column3","column4","column5"
"20171108","120909470","0cdd3c3acdb7cfa107286565c044c5a0f1e58268f6f10e7e3415ff84942e577d","12","101 "
"20171109","122715740","0a7fb9f6bb7a180f2fd9429b0fbd1e7b0a83597b6a64aa6a123cef3e84700fe3","11","101"
Bacon Bits appears to have the correct methodology minus one part. The ForEach loop in your original example does not modify the original variable. Also, it appears the column you want to modify is not 'Column3', but 'Column #2' as the headers begin at zero. I'll repeat the function provided in Bacon Bits's suggestion.
function Get-StringHash {
[CmdletBinding()]
param (
[Parameter(Mandatory = $true, ValueFromPipeline = $true, Position = 0)]
[String[]]
$String,
[Parameter(Position = 1)]
[ValidateSet('SHA1', 'MD5', 'SHA256', 'SHA384', 'SHA512')]
[String]
$HashName = 'SHA256'
)
process {
$StringBuilder = [System.Text.StringBuilder]::new(128)
[System.Security.Cryptography.HashAlgorithm]::Create($HashName).ComputeHash([System.Text.Encoding]::UTF8.GetBytes($String)) | ForEach-Object {
[Void]$StringBuilder.Append($_.ToString("x2"))
}
$StringBuilder.ToString()
}
}
I would suggest for the substitution:
$csv = Import-Csv .\results.csv | Select-Object *,#{n='Column #2';e={Get-StringHash $_.'Column #2'}} -ExcludeProperty 'Column #2'
$CSV | Export-Csv .\myobfuscated.csv -NoTypeInformation
This will put the 'Column #2' last in the CSV. You can simply list them explicitly if you need it to appear in the same order, e.g.:
Select-Object 'Column #0','Column #1',#{n='Column #2';e={Get-StringHash $_.'Column #2'}},'Column #3'

PowerShell to Create Custom Object from Existing Object

I'm attempting to create a custom object from an existing object. However, I can't seem to figure out the best way to go about doing this.
My goal is to create a custom object for each computer name that will have all of the values from Application Name.
I'm not sure if I should use a foreach or a where clause to make this happen and I'm also not sure the correct syntax to use either.
Thanks in advance for any help with this.
The data in the existing object looks like:
Computer Name, IP Address, Application Name
host1,10.10.10.10,firefox
host1,10.10.10.10,chrome
host1,10.10.10.10,internet explorer
host2,11.11.11.11,firefox
host2,11.11.11.11,chrome
host2,11.11.11.11,opera
Code Example:
foreach ($global:ComputerName in $global:SNWReportObject."Computer name"){
if ($_ -eq $global:ComputerName) {
Add-Member -InputObject $global:NEWReportObject -MemberType NoteProperty -Name ApplicationName -Value $global:SNWReportObject."Application name"
}
}
Results I'm hoping for:
Computer Name, Application Name
host1,firefox,chrome,internet explorer
host2,firefox,chrome,opera
Edit:
So your actual case was a bit weird since we're working with arrays of strings in a custom object rather than a hash table. The following code snippet will turn your object into a [HashTable] that is easy to work with. Assumption: all the input data is this well-formed
<# Example.csv
Computer name,IP-Address,Application
host1,10.10.10.10,firefox
host1,10.10.10.10,chrome
host1,10.10.10.10,internet explorer
host2,11.11.11.11,firefox
host2,11.11.11.11,chrome
host2,11.11.11.11,opera
#>
$Csv = Import-Csv -LiteralPath C:\Temp\Example.csv
[HashTable]$Hash=#{}
For ($i = 0; $i -lt ($Csv.'Computer name').Count; $i++)
{
If ($Hash.ContainsKey($Csv.'Computer name'[$i]))
{
$Hash.($Csv.'Computer name'[$i]).Application += $Csv.Application[$i]
Continue
}
$Hash.($Csv.'Computer name'[$i]) = #{
Application = #($Csv.Application[$i])
}
}
Now your members are accessible like so: $Hash.Host1.Application
To get your desired output:
Set-Content -Path C:\Temp\file.csv -Value 'Computer Name,Application Name'
$Hash.Keys |
ForEach-Object
{
Add-Content -Path C:\temp\file.csv -Value "$_,$($Hash.$_.Application -join ',')"
}
When working with a [PSCustomObject], your members are quite limited in what you can do. Here are the standard members (to ignore):
TypeName: System.Management.Automation.PSCustomObject
Name MemberType Definition
---- ---------- ----------
Equals Method bool Equals(System.Object obj)
GetHashCode Method int GetHashCode()
GetType Method type GetType()
ToString Method string ToString()
Knowing this, you can parse through the rest to copy:
$Comp = #('Equals','GetHashCode','GetType','ToString'
[HashTable]$CopyObject=#{}
($Object | Get-Member).Name |
ForEach-Object {
If ($_ -notin $Comp) {$CopyObject.$_ = $Object.$_}
}

Compare Two Objects Properties and Obtain Missing Items

I'm attempting to compare two object properties called "name" and that contain the same type of data, in this case server host names. The objects are not identical AKA do not have the same properties. In short, I'm attempting to compare two lists of server names and determine where (which object) they are missing from.
I'm looking to find the items (server host names) that are missing from each object. When something is found I'm hoping to obtain all related properties for the item in the given object that it was found in. I can do the compare-object successfully, but don't know how to get the results I'm looking for.
I'm thinking two new objects could be created for each, that list the items that were not found in the other object maybe? Or do I somehow reference the previous objects with the output from compare-object and produce some formatted output?
This code currently produces a blank file.
Data Format:
$SNObject:
name,ip,class
server-place.com,10.10.10.10,windows server
$QRObject:
name,date,ip,general,device type
server-place1.com,11.11.11.11,random info,linux server
Code Example:
$compare = compare-object $SNObject $QRObject -property Name |
foreach {
if ($_.sideindicator -eq '<=')
{$_.sideindicator = $PathToQRReport }
if ($_.sideindicator -eq '=>')
{$_.sideindicator = $PathToSNReport}
}
$compare |
select #{l='Value';e={$_.InputObject}},#{l='File';e={$_.SideIndicator}} |
Out-File -FilePath C:\Temp\MissingOutputs1.txt
Ahh... just thought of an alternative that may give you exactly what you're looking for but in a slightly different way:
## Join both arrays into single array and then group it on the property name that has a shared value, 'name'
$all = #()
$group = #($SNObject + $QRObject)
$group | Group-Object -Property name | % {
## Create a custom object that contains all possible properties plus a directionality indicator ('source')
$n = New-Object PSObject -Property #{
'name' = ''
'date' = ''
'ip' = ''
'general' = ''
'platform' = ''
'source' = ''
}
if ($_.Count -eq 1) {
## Loop through the grouped results and determine their source and write properties based off of their source
foreach ($i in $_.Group) {
if (#($i | Get-Member -MemberType NoteProperty | Select-Object -ExpandProperty name) -contains 'date' ) {
## This value came from $QRObject which apparently is the only dataset that contains a date property
$n.source = 'QRObject'
$n.date = $i.date
$n.general = $i.general
$n.platform = $i.'device type'
} else {
## This object does not contain the 'date' property, therefore it came from $SNObject
$n.source = 'SNObject'
$n.platform = $i.class
}
## write out common properties
$n.name = $i.name
$n.ip = $i.ip
## add the custom PSObject back to a master array with all formatted properties
$all += $n
}
}
}
$all | out-whereever-you-want

Resources