I have two csv files:
ipaddress,port
10.140.11.1,80
10.140.11.2,80
ipaddress,port
10.140.11.1,80
10.140.11.2,8008
The question is how to compare the files in powershell. I have already tried this:
$file1 = import-csv "csvfile1.csv"
$file2 = import-csv "csvfile2.csv"
Compare-Object $file1 $file2 -IncludeEqual
The result is as those two files were equal.
It works as expected if I specify the particular property, e.g:
Compare-Object $file1 $file2 -IncludeEqual -Property port
How to compare the csv files without specifying the properties. Let's say I would like to compare all properties in the csv file.
You can obtain the list of CSV column properties via Get-Member -MemberType NoteProperty, then pass that list to Compare-Object.
# get list of CSV properties
$props1 = $file1 | gm -MemberType NoteProperty | select -expand Name | sort | % {"$_"}
$props2 = $file2 | gm -MemberType NoteProperty | select -expand Name | sort | % {"$_"}
# first check that properties match (can omit this step if you know for sure they will be)
if(Compare-Object $props1 $props2)
{
throw "Properties are not the same! [$props1] [$props2]"
}
# pass properties list to Compare-Object
else
{
Compare-Object $file1 $file2 -Property $props1
}
The answer by latkin will not work.
You will get the following exception:
Compare-Object : Cannot convert System.Management.Automation.PSObject to one of the following types {System.String, System.Management.Automation.ScriptBlock}.
At line:8 char:19
+ Compare-Object <<<< $file1 $file2 -Property $props1
+ CategoryInfo : InvalidArgument: (:) [Compare-Object], NotSupportedException
+ FullyQualifiedErrorId : DictionaryKeyUnknownType,Microsoft.PowerShell.Commands.CompareObjectCommand
It seems that one cannot pass a variable for -Property. It has to be a comma-seperated list of NoteProperties and it cannot be enclosed in single or double quotes.
I've been looking for a way to do this same thing and I still haven't found a way...
Related
I'm trying to create a sorting function to sort my objects into three different lists. Each object contains the names of servers (among other things), under the "Name" column. I'm trying to create three lists. One that contains server names that appear on both objects, one that contains server names that only appear on the txtFile object, and one that contains server names that only appear on the csvFile object. Here is what I have so far:
If ($txtFile.Name -contains $csvFile.Name) {
$onBothLists += $csvFile.Name
}
ElseIf ($txtFile.Name -notcontains $csvFile.Name) {
$onlyOnTxtFile += $txtFile.Name
}
ElseIf ($csvFile.Name -notcontains $txtFile.Name) {
$onlyOnCsvFile += $csvFile.Name
}
My issue is that when I run this, $onBothLists and $onlyOnTxtFile populate, while $onlyOnCsvFile does not. However, when I run a Compare-Object for them, it outputs three lists exactly how I expect it to. Is my logic wrong here?
You can loop over the lists yourself, but Compare-Object already does all the iterating and comparison you need. You just have to filter the results afterward. This is made easy by Group-Object.
Compare-Object returns a list of objects each containing 2 properties: the original object (.InputObject) and an indicator of which list the resulting object was found in (.SideIndicator), which shows == for objects in both, or rockets for a direction <= and =>.
By grouping on the side indicator and giving the group results as a [hashtable] we can easily index into that table by side indicator to get the results desired:
$list1 = echo serverA serverB serverC serverD serverE
$list2 = echo serverD serverE serverF serverG serverH
$grouped = Compare-Object -ReferenceObject $list1 -DifferenceObject $list2 -IncludeEqual |
Group-Object -Property SideIndicator -AsHashTable -AsString
# both
$grouped['=='].InputObject
# in list1
$grouped['<='].InputObject
# in list2
$grouped['=>'].InputObject
For just the ones that are the same, you can do this:
Compare-Object -ReferenceObject $list1 -DifferenceObject $list2 -IncludeEqual -ExcludeDifferent
To build on my answer to your previous question:
## Q:\Test\2018\12\21\SO_53886784.ps1
$csvFile = Import-Csv .\sample.csv
$txtFile = Import-csv .\sample.txt -Header Name
$newCsv = Compare-Object -Ref $csvFile -Dif $txtFile -Property Name -IncludeEqual
$onBothLists = $newCsv | Where-Object SideIndicator -eq '==' | Select-Object -Expand Name
$onlyOnTxtFile = $newCsv | Where-Object SideIndicator -eq '=>' | Select-Object -Expand Name
$onlyOnCsvFile = $newCsv | Where-Object SideIndicator -eq '<=' | Select-Object -Expand Name
Sample output:
> $onBothLists
wddg9028
htew804
> $onlyOnTxtFile
test1234
> $onlyOnCsvFile
other321
I've got a script that searches for a string ("End program" in this case). It then goes through each file within the folder and outputs any files not containing the string.
It works perfectly when the phrase is hard coded, but I want to make it more dynamic by creating a text file to hold the string. In the future, I want to be able to add to the list of string in the text file. I can't find this online anywhere, so any help is appreciated.
Current code:
$Folder = "\\test path"
$Files = Get-ChildItem $Folder -Filter "*.log" |
? {$_.LastWriteTime -gt (Get-Date).AddDays(-31)}
# String to search for within the file
$SearchTerm = "*End program*"
foreach ($File in $Files) {
$Text = Get-Content "$Folder\$File" | select -Last 1
if ($Text | WHERE {$Text -inotlike $SearchTerm}) {
$Arr += $File
}
}
if ($Arr.Count -eq 0) {
break
}
This is a simplified version of the code displaying only the problematic area. I'd like to put "End program" and another string "End" in a text file.
The following is what the contents of the file look like:
*End program*,*Start*
If you want to check whether a file contains (or doesn't contain) a number of given terms you're better off using a regular expression. Read the terms from a file, escape them, and join them to an alternation:
$terms = Get-Content 'C:\path\to\terms.txt' |
ForEach-Object { [regex]::Escape($_) }
$pattern = $terms -join '|'
Each term in the file should be in a separate line with no leading or trailing wildcard characters. Like this:
End program
Start
With that you can check if the files in a folder don't contain any of the terms like this:
Get-ChildItem $folder | Where-Object {
-not $_.PSIsContainer -and
(Get-Content $_.FullName | Select-Object -Last 1) -notmatch $pattern
}
If you want to check the entire files instead of just their last line change
Get-Content $_.FullName | Select-Object -Last 1
to
Get-Content $_.FullName | Out-String
Im getting the names of these computers and putting them into an array. Now what i want to do is to convert them into a string array to be able to check which policy they are on using a Get-ADComputer for loop or using a foreach loop (Can you recommend which one to use)
$global:arrComputers = #()
$computerStrings = Get-ADComputer -Filter 'SamAccountName -like "*Name*"' | Select -Expand Name
foreach ($line in $computerStrings)
{
$a = $line.ToString()
$b = $a.split()
$temp = #{}
$temp = New-Object object
$temp | Add-Member -MemberType "noteproperty" -Name Name -Value $b[0]
$global:arrComputers += $temp
}
$global:arrComputers
This is the command i want to run to check the policy they are under
Get-ADComputer "Name" -Properties MemberOf | %{if ($_.MemberOf -like "*POLICY_NAME*") {Write-Host "ON"} else {Write-Host "NOT ON"}}
I have tested both blocks of code and they are working the only problem im having is turning that array into a string array. I also tried the ToString() To be able to loop through it with the Get-ADComputer "Name"
"memberOf" property in objects returned by "Get-ADComputer" returns a list of strings containing Distinguished Name of each group this computer is a member of.
Therefore, I assume when you say "This is the command i want to run to check the policy they are under", you are referring to a group membership that a group policy is targeting right?
Below code then will do it:
$computers = #();
Get-ADComputer -Filter * -Properties Name,MemberOf | %{if ($_.MemberOf -like "*computer_group_name*") { $computers += $_.Name } }
Explanation:
First line, define an array $computers
Second line, query AD for computer object properties Name,MemberOf
then, $_.MemberOf contains group name in string, add Name property(string) to array of strings you defined on line 1
I have the following in a script:
$Loc = Read-Host "Please select a location number"
$ImpCSV = Import-Csv -Delimiter ";" -Path "file.csv" -Header loc,add | Where-Object {$_.loc -eq "$Loc"}
Foreach ($CIM in $ImpCSV) {$LocInfo = $CIM}
This seems to create some sort of hash/array/something variable for $Loc=1. If I enter:
$LocInfo.add
It gives me the value of that key(?) as it should in a hash table, but if I do:
$LocInfo
I get:
loc : [1]
add : [some value]
Which is not the usual hash output. This is what one should look like:
Name Value
---- -----
loc [1]
add [some value]
The issue this is first causing me is that if I try to edit an entry using $LocInfo.Set_Item("add", "HI"), it does nothing. Also, I created a table to display those variables creating new-objects, but the new -property command won't take $LocInfo.add, returning a "#{loc=1; add=some value}" if not giving me a syntax error. What I had to do was:
Set-Variable -Name add -Value $LocInfo.add
This sets add to [some value] and then I can put that in the new-object field without any issue.
This is additionally problematic because my script loops so they can add another location (loc) number (let's say 2) and pull up that location's data as necessary. If the new location they are searching has say the add (Address) missing, instead of creating a blank $LocInfo.add and then a blank $add, it skips it, meaning when the new-object is created again, it uses the $add from the previous run, [some value]; ie both loc 1 and 2 have the same $add value.
What that heck did I create, and how can I do anything with it? Is there a better way to pull variables from a CSV file that won't screw me up like this? Did I break Powershell? I've looked all over and I can't find this kind of variable arrangement.
This is the error I get when I try to do a Set_Item:
Method invocation failed because [System.Management.Automation.PSCustomObject] does not contain a method named 'Set_Item'.
At line:1 char:1
+ $LocInfo.Set_Item("add", "HI")
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (Set_Item:String) [], RuntimeException
+ FullyQualifiedErrorId : MethodNotFound
I've created objects using New-Object, and even those display like a hash table, ie with a header and multiple columns depending on the object size.
First thing: If you are ever not sure about the type of an object you can just check and object's type with GetType(). Import-Csv command returns a System.Object[] where a command like this #{Letter="Test"}.GetType().FullName would return System.Collections.Hashtable. .FullName returns the just the strings I am using in these examples.
Appears that you are trying to collect your output with the Where-Object using this line Foreach ($CIM in $ImpCSV) {$LocInfo = $CIM} The couple of issues I see with that are your keep overwriting $LocInfo with $CIM. At the end of the loop you will only have one variable. If you are trying to keep all of them would need to append each item and declare $LocInfo as an array.
$LocInfo = #()
....
Foreach ($CIM in $ImpCSV) {$LocInfo += $CIM}
But unless you have more code that you are not showing this appears redundant as you would have $LocInfo and $ImpCSV populated with the same data. You could just do away with the for loop altogether and just do this.
$Loc = Read-Host "Please select a location number"
$ImpCSV = Import-Csv -Delimiter ";" -Path "file.csv" -Header loc,add | Where-Object {$_.loc -eq "$Loc"}
I have changed nothing except remove the last line.
Does converting the result of import-csv into custom object like this help?
$ImpCSV = Import-Csv -Delimiter ";" -Path "file.csv" -Header loc,add | Where-Object {$_.loc -eq "$Loc"} | %{[pscustomobject]#{LOC=$_.LOC;ADD=$_.ADD}}
or
$ImpCSV = Import-Csv -Delimiter ";" -Path "file.csv" -Header loc,add | Where-Object {$_.loc -eq "$Loc"} | %{new-object -TypeName PSCustomObject -Property #{LOC=$_.LOC;ADD=$_.ADD}}
Import-Csv reads a CSV file and transforms it into a list of custom objects with the CSV columns as properties. However, objects with just 2 properties should be displayed in table format by default:
PS C:\> cat 'C:\test.csv'
1,foo
2,bar
3,baz
PS C:\> $csv = Import-Csv 'C:\test.csv' -Header loc,add | ? {$_.loc -eq '2'}
PS C:\> $csv
loc add
--- ---
2 bar
You should only get output in list format if you pipe the object(s) into the Format-List cmdlet:
PS C:\> $csv | Format-List
loc : 2
add : bar
or if the objects have 5 or more properties:
PS C:\> cat 'C:\test2.csv'
1,foo,a,a,a
2,bar,b,b,b
3,baz,c,c,c
PS C:\> Import-Csv 'c:\test2.csv' -Header loc,add,f3,f4 | ? {$_.loc -eq '2'}
loc add f3 f4
--- --- -- --
2 bar b b
PS C:\> Import-Csv 'c:\test2.csv' -Header loc,add,f3,f4,f5 | ? {$_.loc -eq '2'}
loc : 2
add : bar
f3 : b
f4 : b
f5 : b
You can enforce tabular output for objects that would normally be displayed in list format by piping them into the Format-Table cmdlet:
PS C:\> Import-Csv 'C:\test2.csv' -Header loc,add,f3,f4,f5 |
>> ? {$_.loc -eq '2'} | Format-Table
>>
loc add f3 f4 f5
--- --- -- -- --
2 bar b b b
Note, however, that formatting cmdlets like Format-List or Format-Table are intended for displaying data. Don't use them if you want to further process the object(s).
I am iterating through a directory full of sub directories, looking for the newest file at each level.
The code below does this, but I need to be able to add each line/loop of the iterator to an array so that at the end I can output all the data in tabular format for use in Excel.
Any advice on how I can do this?
$arr = get-childItem -Path "\\network location\directory" | select FullName
$res = #()
foreach($fp in $arr)
{
get-childItem -Path $fp.FullName | sort LastWriteTime | select -last 1 Directory, FullName, Name, LastWriteTime
}
Here's a one-liner for you, split onto multiple lines for readability with the backtick escape character. You can copy paste this and it will run as is. The csv file will be created in the folder where you run this from.
dir -rec -directory | `
foreach {
dir $_.fullname -file | `
sort -Descending lastwritetime | `
select -first 1
} | `
export-csv newestfiles.csv
dir is an alias for get-childitem. foreach is an alias for foreach-object. %, gci and ls are even shorter aliases for get-childitem. Note that I am avoiding storing things in arrays, as this is doubling the work required. There is no need to enumerate the folders, and then enumerate the array afterwards as two separate operations.
Hope this helps.
If I understand you correctly, you just need to pipe the results into $res. So adding | %{$res += $_} should do the trick
$arr = get-childItem -Path "\\network location\directory" | select FullName
$res = #()
foreach($fp in $arr)
{
get-childItem -Path $fp.FullName | sort LastWriteTime | select -last 1 Directory, FullName, Name, LastWriteTime | % {$res += $_}
}
$res | % {write-host $_}