I'm trying to transform a date format from "PT10H24M30S" to "10:24:30".
I created the following function:
function PTTime([string] $time){
$pattern = "(\d{2})+"
$matches = $time | Select-String -Pattern $pattern -AllMatches
$newFormat += $matches.Matches | ForEach {$_.Value}
$newFormat = $newFormat -Replace " ", ":"
return $newFormat
}
However, if I call the function I got the output
10 24 30
or
10
24
30
But in the other hand, If I execute the function commands 1 at a time on the command line I got the correct output when I print $newFormat "10:24:30"
I realized that $newFormat was a different type than $returnValue
PS was a bit miss leading here, because when I called $newFormat | Get-Member or $returnValue | Get-Memeber
the output type always is TypeName: System.String.
However, if a do $newFormat.GetType() I got System.String while if I call $returnValue.GetType() I got System.Array[].
I updated the function and now is working as expected
function PTTime([string] $time){
$pattern = "(\d{2})+"
$matches = $time | Select-String -Pattern $pattern -AllMatches
$newFormat = $matches.Matches.Value
return $newFormat -Join ":"
}
Related
$space =("`r`n")
$data = #(Get-Content C:\Users\user1\Desktop\ma.txt)
$Array = ($Data.Split($space)).Split($space)
$pos1 = $Array[0][0]+$Array[0][1]
$pos2 = $Array[1][0]+$Array[1][1]
$pos3 = $Array[2][0]+$Array[2][1]
#$pos1
#$pos2
#$pos3
$zahl1 = $Array[0][5]+$Array[0][7]+$Array[0][9]
$zahl1
PowerShell 7.2
txt1.txt has the text:
x1 = 2 + 3
x2 = 8 / 4
x3 = 1 - 4
i want the results (from x1,x2,x3) to be saved at txt2.txt with a command in Terminal.
I tried whith Arrays,
but i only get :2+3 instead of 5
Any thoughts?
You could use Invoke-Expression for this, but read the warning first
Get-Content -Path text1.txt | Where-Object {$_ -match '\S'} | ForEach-Object {
$var,$calculation = ($_ -split '=').Trim()
'{0} --> {1}' -f $var, (Invoke-Expression -Command $calculation)
} | Set-Content -Path text2.txt
This is an attempt of a more secure version, that matches only mathematical expressions, so users cannot run arbitrary code through Invoke-Expression:
Get-Content text1.txt |
Select-String '^\s*(\S+)\s*=([\d\.+\-*/%\(\)\s]+)$' |
ForEach-Object {
$var = $_.Matches.Groups[ 1 ].Value
$expression = $_.Matches.Groups[ 2 ].Value
$result = Invoke-Expression $expression
"{0} = {1}" -f $var, $result
} |
Set-Content text2.txt
The Select-String cmdlet uses a regular expression to match only lines that are considered "safe". Within the RegEx there are two groups defined to split the line into variable (1) and calculation (2) sub strings. These are then extracted via $_.Matches.Groups.
RegEx breakdown:
Pattern
Description
^
line start
\s*
zero or more whitespace characters
(
start 1st capturing group
\S+
one or more non-whitespace characters
)
end 1st capturing group
\s*
zero or more whitespace characters
=
literal "="
(
start 2nd capturing group
[
start list of allowed characters
\d\.+\-*/%\(\)\s
digits, dot, math ops, parentheses, whitespace
]
end the list of allowed characters
+
one or more chars (from the list of allowed characters)
)
end 2nd capturing group
$
line end
I have this function that gets the content of a json file.
I'm having some (to me) unexpected behavior trying to pipe this to ConvertTo-Json and Foreach-Object
Function GetConfigJson
{
$ConfigPath = "pathtomyjsonfile.json"
return Get-Content $ConfigPath | Out-String
}
The json is formatted like [{"key1":"value1"}, {"key2":"value2"}, ...]
To test the behavior I did the following:
$a = 0;
GetConfigJson | ConvertFrom-Json | ForEach-Object { $a++ };
$b = 0;
ConvertFrom-Json GetConfigJson | ForEach-Object { $b++ };
$c = 0;
ConvertFrom-Json (GetConfigJson) | ForEach-Object { $c++ };
$d = 0;
(ConvertFrom-Json (GetConfigJson)) | ForEach-Object { $d++ };
Write-Host "Test1: $a | Test2: $b | Test3: $c | Test4: $d";
Out of these only Test4 prints the expected number, Test1 and Test3 print 1 and Test2 gets an error: ConvertFrom-Json : Invalid JSON primitive: GetConfigJson.
Why do I need the parentheses around the ConvertFrom-Json for it to actually get piped as an array of objects?
(The parentheses around the function name GetConfigJson is more acceptable - but I still wonder why I need it there?)
It might help to take a look at the types of the output from each example - see below for a breakdown.
Helper Functions
I'm using these the two helper functions in the sections below. Note - I'm guessing your config has an array at the root as that seems to reproduce the issue, but feel free to update your question if that's not true.
function GetConfigJson
{
return "[{""name"":""first""}, {""name"":""second""}]"
}
function Write-Value
{
param( $Value )
write-host $Value.GetType().FullName
write-host (ConvertTo-Json $Value -Compress)
}
And then using your examples:
Example 1
# example 1a - original example
PS> $a = 0
PS> GetConfigJson | ConvertFrom-Json | ForEach-Object { $a++ };
PS> $a
1
# example 1b - show return types and values
PS> GetConfigJson | ConvertFrom-Json | foreach-object { Write-Value $_ }
System.Object[]
{"value":[{"name":"first"},{"name":"second"}],"Count":2}
ConvertFrom-Json returns an array object with two entries, but Foreach-Object only runs once because it iterates over the single array object, not the 2 items in the array.
Example 2
# example 2a - original example
PS> $b = 0;
PS> ConvertFrom-Json GetConfigJson | foreach-object { $b++ }
ConvertFrom-Json : Invalid JSON primitive: GetConfigJson.
At line:1 char:1
+ ConvertFrom-Json GetConfigJson | foreach-object { $b++ }
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [ConvertFrom-Json], ArgumentException
+ FullyQualifiedErrorId : System.ArgumentException,Microsoft.PowerShell.Commands.ConvertFromJsonCommand
# example 2b - show parameter types and values
PS> Write-Value GetConfigJson
System.String
"GetConfigJson"
ConvertFrom-Json throws an exception because PowerShell is treating GetConfigJson as a literal string, but "GetConfigJson" obviously isn't valid json, hence the exception.
Example 3
# example 3a - original example
PS> $c = 0;
PS> ConvertFrom-Json (GetConfigJson) | ForEach-Object { $c++ };
PS> $c
1
# example 3b - show parameter types and values
PS> ConvertFrom-Json (GetConfigJson) | ForEach-Object { Write-Value $_ };
System.Object[]
{"value":[{"name":"first"},{"name":"second"}],"Count":2}
This uses the Grouping Operator ( ... ) around GetConfigJson, so PowerShell evaluates GetConfigJson as a call to a function instead of taking it as a literal string. It first executes the GetConfigJson expression and then passes the result of that as a parameter into ConvertFrom-Json. However, it's still iterating over the single array object rather than over the items, so the foreach-object only runs once.
Example 4
# example 4a - original example
PS> $d = 0;
PS> (ConvertFrom-Json (GetConfigJson)) | ForEach-Object { $d++ };
PS> $d
2
# example 4b - show parameter types and values
PS> (ConvertFrom-Json (GetConfigJson)) | ForEach-Object { Write-Value $_ };
System.Management.Automation.PSCustomObject
{"name":"first"}
System.Management.Automation.PSCustomObject
{"name":"second"}
We're using the grouping operator twice here - once around GetConfigJson to evaluate that as an expression as opposed to a string, and once around the whole ConvertFrom-Json (GetConfigJson). The outer ( ... ) causes PowerShell to "unroll" the single array object and emits its items into the pipeline consumed by Foreach-object. This means ForEach-Object iterates over the items and we see two separate values written out by ``Write-Value```
Summary
You managed to hit a lot of PowerShell quirks with this question - hopefully this answer helps understand what they're all doing and why you see the behaviour you do.
Update for PowerShell 7
Per the comment below from #mklement0, the behaviour of ConvertFrom-Json changes from version 7 onwards - it doesn't enumerate arrays by default, and requires -NoEnumerate to opt out.
E.g., '[ 1, 2 ]' | ConvertFrom-Json | Measure-Object now reports 2 in v7+, whereas -NoEnumerate is required to get the v6- behaviour: '[ 1, 2 ]' | ConvertFrom-Json -NoEnumerate | Measure-Object (reports 1).
Bond.out file example (looking to replace what is highlighted):
Out.csv file (data to be used):
Code:
#set paths up
$filepath= 'C:\folder\path\bond.out'
$filepath2= 'C:\folder\path\temp.txt'
$Ticklist='C:\folder\path\tick.txt'
$ratelist='C:\folder\path\rate.txt'
#Import needed data from an excel file which creates and array
$csv = Import-CSV C:\folder\path\RateIDTable.csv | Where { $_.'Rate' -ne "" } | Export-Csv C:\folder\path\out.csv -NoTypeInformation
$bond = Import-CSV C:\folder\path\out.csv | select -Property TickerID, Rate
#Put array from Excel file into two text files
$Tick = $bond | foreach-object {$_.TickerID} | set-content $Ticklist
$replace = $bond | foreach-object {$_.rate} | set-content $Ratelist
#Create two separate arrays from the new text files
$Tickdata = (Get-content $Ticklist ) -join ','
foreach ($t in $Tickdata)
{
$t = $t -split(",")
$First = $t[0]}
$Ratedata = (Get-content $Ratelist ) -join ','
foreach ($r in $Ratedata)
{
$r = $r -split(",")
$First = $r[0]}
#Get main file to search (bond.out) and search for the word that is in the first line from "t" array file
$data = Select-String $filepath -pattern $t[0] | Select-Object -ExpandProperty Line
$data
#Once found, split the line, replace the rate on the 3rd line with the rate in the first line from the "r" array file, the put the line back to together
$split=$data.split("{|}")
$split[3]=$r[0]
$join = $split -join "|"
$join
#Put the updated line back into the "bond.out" file from whence it came
(get-content $filepath) -replace($data,$join) | set-content $filepath
#computer says no :(
Output:
As you can see, it actually replaces the rate and puts it all back like I need it to. But that last line doesn't seem to work. Instead I get the file back like so:
It appears as though it is repeating the same line from the $join parameter and adding letters to the beginning of each iteration.
I believe it has something to do with the '|' at the end of the line, and remember reading something about marking the beginning and end of lines some time ago, but can't find it anywhere.
Here's an idea. Instead of using regular expressions ...
The Import-Csv command has a -Delimiter parameter. Can you just import bond.out as a "CSV" (but with a pipe delimiter), and update it just like you would a CSV file?
Pseudo-code
### Convert bond.out to objects
$BondOut = Import-Csv -Delimiter '|' -Path $FilePath
### Get the line you want to update
$LineToUpdate = $BondOut.Where({ $PSItem.TickerID -eq 'BBG0019K2QZ5' })
### Update the Rate property from your source (out.csv)
$LineToUpdate.Rate = $SomeSource.Rate
### Export the modified objects to a new bond.out.modified file
$BondOut | Export-Csv -Delimiter '|' -Path 'bond.out.modified' -NoTypeInformation
As per PetSerAI's clue:
#set paths up
$filepath= 'C:\folder\path\bond.out'
$filepath2= 'C:\folder\path\temp.txt'
$Ticklist='C:\folder\path\tick.txt'
$ratelist='C:\folder\path\rate.txt'
#Import needed data from an excel file which creates and array
$csv = Import-CSV C:\folder\path\RateIDTable.csv | Where { $_.'Rate' -ne "" } | Export-Csv C:\folder\path\out.csv -NoTypeInformation
$bond = Import-CSV C:\folder\path\out.csv | select -Property TickerID, Rate
#Put array from Excel file into two text files
$Tick = $bond | foreach-object {$_.TickerID} | set-content $Ticklist
$replace = $bond | foreach-object {$_.rate} | set-content $Ratelist
#Create two separate arrays from the new text files
$Tickdata = (Get-content $Ticklist ) -join ','
foreach ($t in $Tickdata)
{
$t = $t -split(",")
}
$Ratedata = (Get-content $Ratelist ) -join ','
foreach ($r in $Ratedata)
{
$r = $r -split(",")
}
#Get main file to search (bond.out) and search for the word that is in the first line from "t" array file
###Replace all pipes with a comma
(get-content $filepath) -replace('\|', ',') | set-content $filepath
$data = Select-String $filepath -pattern $t[0] | Select-Object -ExpandProperty Line
$data
#Once found, split the line, replace the rate on the 3rd line with the rate in the first line from the "r" array file, the put the line back to together
$split=$data.split("{,}")
$split[3]=$r[0]
$join = $split -join ","
#Put the updated line back into the "bond.out" file from whence it came
###change all commas back to pipes
(get-content $filepath) -replace($data,$j) | set-content $filepath
(get-content $filepath) -replace(',', '|') | set-content $filepath
#computer says yay :D
My goal is to take two objects (created by importing CSVs) that have host names and compare one list to the other and show what's missing from each.
Before I can do the comparison I need to manipulate the host names stored within the object(s). The first step is to use regular expressions to remove (-replace) unnecessary text and then set all host names to lowercase (ToLower()).
I'm not very proficient with modifying existing objects and keeping them "intact", so I'm hoping someone could help me with this.
Here's an example of the data stored within the CSV. The header is on line 7 and each line of data is stored like:
...
7 "name","IP","OSType"
8 "WCSMserver.com","10.10.10.10","OSX"
9 "SERVER2.com","11.11.11.11","Windows"
10 "windowsserver # SERVER2.com","11.11.11.13","Windows"
11 "winner.comSERVER2.com","11.11.11.12","Windows"
...
Here's an example of what I'm trying to do so far (just replacing the name property values):
function ReadExcelReport() {
$global:ConvertedQReportTest = $PSScriptRoot + "\" + "AllSources.csv"
$global:QReportObject = Get-Content -Path $global:ConvertedQReportTest |
Select-Object -Skip 7 |
ConvertFrom-Csv
}
ReadExcelReport
$global:QReportObject.name = $global:QReportObject.name | ForEach-Object {
#($global:QReportObject.name)
$_ -replace 'WCSM \- ' `
-replace '.*?# '`
-replace '.*?#'`
-replace '.*?\:\:.*?'`
-replace '\.cooper\.winner\.com'`
-replace '\.winner\.com'
}
By doing $global:QReportObject.name | ForEach-Object you loop the names of the objects and not the objects.
I've simplified your script a bit (for readability):
$csv = #"
"name"
"WCSMserver-remove this-com"
"SERVER2.com","11.11.11.11"
"windowsserver-remove this-"
"winner.comSERVER2.com"
"#
$global:QReportObject = $csv | ConvertFrom-Csv
$global:QReportObject | Out-Default
$global:QReportObject | ForEach-Object {
$_.name = $_.name -replace '-remove this-'
$_.name = $_.name.ToLower()
}
$global:QReportObject | Out-Default
This will output:
name
----
WCSMserver-remove this-com
SERVER2.com
windowsserver-remove this-
winner.comSERVER2.com
name
----
wcsmservercom
server2.com
windowsserver
winner.comserver2.com
Input:
1116559 P1303251287 20130325225906CD 13013822 1 0000
1104220 P1303250282 20130325070119CD 1 0000
1064743 P1303251094 20130325191600CD 0 0000
1100819 P1303250369 20130325091722CD 0 0000
1101405 P1303250051 20130325010740CD 2 0000
What I get from my attempt:
$lines = Get-Content "filenamehere.txt"
ForEach ($x in $lines) {
$y = "$($x[0..9] -join '')|$($x[10..23] -join '')|$($x[24..42] -join '')|
$($x[43..53] - join '')|$($x[54..57] -join '')|$($x[58..61] -join '')|
$($x[126..138] -join '')"
$z = $y -join '|'
Write-Output $z | Out-File -FilePath "foo.txt" -Append}
I get:
1116559 |P1303251287 |20130325225906CD |13013822 |1 |0000|
1104220 |P1303250282 |20130325070119CD | |1 |0000|
1064743 |P1303251094 |20130325191600CD | |0 |0000|
1100819 |P1303250369 |20130325091722CD | |0 |0000|
1101405 |P1303250051 |20130325010740CD | |2 |0000|
I don't mind the trailing spaces as long as I can get into this format. But "Get-Content" parse my data into an array and importing to SQL gives me an error. Question is, how can I convert this into CSV?
Output should be:
1116559|P1303251287|20130325225906CD|13013822|1|0000
1104220|P1303250282|20130325070119CD| |1|0000
1064743|P1303251094|20130325191600CD| |0|0000
1100819|P1303250369|20130325091722CD| |0|0000
1101405|P1303250051|20130325010740CD| |2|0000
I'd do that with -replace
$Regex = '(.{7})\s{3}(.{11})\s{3}(.{16})\s{3}(.{8})\s{3}(.{1})\s{3}(.{4})'
$Replace = '$1|$2|$3|$4|$5|$6'
(Get-Content "filenamehere.txt") -replace $Regex,$Replace |
Set-Content "foo.txt"
Using your sample you can use trim()
$lines = Get-Content "c:\temp\filenamehere.txt"
ForEach ($x in $lines)
{
$y = "$(($($x[0..9] -join '')).trim())|$(($($x[10..23] -join '')).trim())|$(($($x[24..42] -join '')).trim())|$(($($x[43..53] -join '')).trim())|$(($($x[54..57] -join '')).trim())|$(($($x[58..61] -join '')).trim())|$(($($x[126..138] -join '')).trim())"
$z = $y -join '|'
Write-Output $z | Out-File -FilePath "c:\temp\foo.txt" -Append
}
Perhaps it remove too much spaces.
I output
1116559|P1303251287|20130325225906CD|13013822|1|0000|
1104220|P1303250282|20130325070119CD||1|0000|
1064743|P1303251094|20130325191600CD||0|0000|
1100819|P1303250369|20130325091722CD||0|0000|
1101405|P1303250051|20130325010740CD||2|0000|
Which should be better on a CSV point of view.
#echo off
for /F "tokens=1-6" %%a in (input.txt) do (
if "%%f" neq "" (
echo %%a^|%%b^|%%c^|%%d^|%%e^|%%f
) else (
echo %%a^|%%b^|%%c^| ^|%%d^|%%e
)
)
Output:
C:\> test.bat
1116559|P1303251287|20130325225906CD|13013822|1|0000
1104220|P1303250282|20130325070119CD| |1|0000
1064743|P1303251094|20130325191600CD| |0|0000
1100819|P1303250369|20130325091722CD| |0|0000
1101405|P1303250051|20130325010740CD| |2|0000
Using a ConvertFrom-FixedLengths function you could just do:
Get-Content "C:\input.txt" |
ConvertFrom-FixedLengths 10,14,19,11,4,4 -Trim |
Foreach { #($_.Column1, $_.Column2, $_.Column3, $_.Column4.PadLeft(8, ' '), $_.Column5, $_.Column6) -Join "|" } |
Out-File -FilePath "c:\output.txt"
Or, of course, if you want to create a csv-file with the | character as delimiter you could just do:
Get-Content "C:\input.txt" |
ConvertFrom-FixedLengths 10,14,19,11,4,4 -Trim |
Select Column1, Column2, Column3, #{ N = "Column4"; E = { $_.Column4.PadLeft(8) } }, Column5, Column6 |
Export-Csv -Path "C:\Output.csv" -NoTypeInformation -Delimiter "|"
Or, to make it even simpler, if you want a csv-file and don't need to pad the fourth column with spaces, you could just skip the `Select´ line in the last sample, making it into:
Get-Content "C:\input.txt" |
ConvertFrom-FixedLengths 10,14,19,11,4,4 -Trim |
Export-Csv -Path "C:\Output.csv" -NoTypeInformation -Delimiter "|"
For a decent solution, you need to handle the content as fixed length fields, the other answers here do that.
If you know only column 4 might be blank, you can bodge it for a one-off script by replacing an 11 character space with a comma (which will do nothing on rows where column 4 has content), then replacing spaces with commas:
Get-Content "data.txt" | % { ($_ -replace "\s{11}", ",") -replace "\s+", "," } > out.txt
Sample output:
1116559,P1303251287,20130325225906CD,13013822,1,0000
1104220,P1303250282,20130325070119CD,,1,0000
1064743,P1303251094,20130325191600CD,,0,0000
1100819,P1303250369,20130325091722CD,,0,0000
1101405,P1303250051,20130325010740CD,,2,0000
Working code..
CD 'C:\\FOLDERPATH\'
$filter = "FILE_NAME_*.txt"
$columns = 11,22,32,42,54
## DO NOT NEED TO REVERSE [array]::Reverse($columns) #too lazy to re-write array after finding out I need to iterate in reverse
$files = get-childitem ./ |where-object {$_.Name -like $filter}
$newDelimiter = '|'
foreach($file in $files)
{
$file
$csvFile = 'C:\\FOLDERPATH\NEW_' + $file.BaseName + '.txt'
if (!(get-childitem ./ |where-object {$_.Name -like $csvFile})) #check whether file has been processed
{
$content | ForEach {
$line = $_
$counter = 0
$columns | ForEach {
$line = $line.Insert($_+$counter, $newDelimiter)
$counter = $counter + 1
}
$line = $line.Trim($newDelimiter)
$line
} | set-content $csvFile
}
}