I need to store data like below for TextToColumns Excel automation.
I need to implement Code-2 or Code-3 or Code-4 is that any way to achieve?
I have more than 350+ data so I cant use Code-1, that's not fair for me.
Code-1: working fine
$var = (1,2),(2,2),(3,2),(4,2),(5,2),(6,2)........(300,2)
$ColumnA.texttocolumns($colrange,1,-412,$false,$false,$false,$false,$false,$true,"|",$var)
Code-2: not Working
$var = #((1,2)..(300,2))
$ColumnA.texttocolumns($colrange,1,-412,$false,$false,$false,$false,$false,$true,"|",$var)
Code-3: not Working
$var = #()
#forloop upto 300
{ $var += ($i,2) }
$ColumnA.texttocolumns($colrange,1,-412,$false,$false,$false,$false,$false,$true,"|",$var)
Code-4: not Working
[array]$var = 1..300 | foreach-object { ,#($_, 2) }
$ColumnA.texttocolumns($colrange,1,-412,$false,$false,$false,$false,$false,$true,"|",$var)
I can't fully explain what happens here but I guess that it is related to the fact that the texttocolumns requires an (deferred) expression rather than an (evaluated) object.
Meaning that the following appears to work for the Minimal, Reproducible Example from #mclayton:
$Var = Invoke-Expression ((1..6 |% { "($_, `$xlTextFormat)" }) -Join ',')
And expect the following to work around the issue in the initial question:
$Var = Invoke-Expression ((1..300 |% { "($_, 2)" }) -Join ',')
Not an answer - just documenting some research to save others some time...
I can repro the issue here with the following code:
$xl = new-object -com excel.application;
$xl.Visible = $true;
$workbook = $xl.Workbooks.Add();
$worksheet = $workbook.Worksheets.Item(1);
$worksheet.Range("A1") = "aaa|111";
$worksheet.Range("A2") = "bbb|222";
$worksheet.Range("A3") = "ccc|333";
$worksheet.Range("A4") = "ddd|444";
$worksheet.Range("A5") = "eee|555";
$worksheet.Range("A6") = "fff|666";
which builds a new spreadsheet like this:
If you then run the following it will parse the contents of column A and put the results into columns B and C:
$range = $worksheet.Range("A:A");
$target = $worksheet.Range("B1");
# XlColumnDataType enumeration
# see https://learn.microsoft.com/en-us/office/vba/api/excel.xlcolumndatatype
$xlTextFormat = 2;
# XlTextParsingType enumeration
# see https://learn.microsoft.com/en-us/office/vba/api/excel.xltextparsingtype
$xlDelimited = 1;
# XlTextQualifier enumeration
# https://learn.microsoft.com/en-us/office/vba/api/excel.xltextqualifier
$xlTextQualifierNone = -4142;
$var = (1,$xlTextFormat),(2,$xlTextFormat),(3,$xlTextFormat),(4,$xlTextFormat),(5,$xlTextFormat),(6,$xlTextFormat);
# parse the values in A1:A6 and puts the values in a 2-dimensional array starting at B1
# see https://learn.microsoft.com/en-us/office/vba/api/excel.range.texttocolumns
$result = $range.TextToColumns(
$target, # Destination
$xlDelimited, # DataType
$xlTextQualifierNone, # TextQualifier
$false, # ConsecutiveDelimiter
$false, # Tab
$false, # Semicolon
$false, # Comma
$false, # Space
$true, # Other
"|", # OtherChar
$var # FieldInfo
);
which then looks like this:
However, if you change the declaration for $var to
$var = 1..6 | % { ,#($_, $xlTextFormat) };
you get the following error:
OperationStopped: The remote procedure call failed. (0x800706BE)
and the Excel instance terminates.
So there's something different about these two declarations:
$var = (1,$xlTextFormat),(2,$xlTextFormat),(3,$xlTextFormat),(4,$xlTextFormat),(5,$xlTextFormat),(6,$xlTextFormat);
$var = 1..6 | % { ,#($_, $xlTextFormat) };
but what that is eludes me :-S
Related
I'm very new to Powershell and what I want to do is to compare two arrays of data and then merge them together to one large array that I can either export to Excel or POST request to a web server using restful API and json.
To do this in Python is simple by using Pandas and adding search data but in Powershell I can't really get it done.
Example: I have 2 arrays
$a1 = #(('Name1', 'Link1', 'URL1'),
('Name2', 'Link2', 'URL2'),
('Name3', 'Link3', 'URL3')
)
$a2 = #(('Name4', 'URL4', 'TEXT4'),
('Name2', 'URL2', 'TEXT1'),
('Name1', 'URL1', 'TEXT2')
)
I want to do a compare $a1 with $a2 and the other way around so I don't miss any values.
Merge them together so I will end up with a $a3 array that look something like this.
$a3 = #(('Name1', 'Link1', 'URL1', 'TEXT1),
('Name2', 'Link2', 'URL2', 'TEXT2),
('Name3', 'Link3', 'URL3', ''),
('Name4', 'URL4', '', 'TEXT4')
)
And as more I dig in to different alternatives as more confused do I get.
I'd recommend to make a collection of (at least) simple objects to work with.
This is almost classic task of combining two different sets of properties by common key. Just convert those arrays-of-arrays to arrays-of-objects-with-properties and combine them next.
Please note that powershell is primary sysadmin's language, there is almost no place for any guesses or chances when processing tasks.
Also you can use almost any external .Net-DLL library in PowerShell (search Deedle as alternative to Pandas)
$a1 = #(('Name1', 'Link1', 'URL1'),
('Name2', 'Link2', 'URL2'),
('Name3', 'Link3', 'URL3')
)
$a2 = #(('Name4', 'URL4', 'TEXT4'),
('Name2', 'URL2', 'TEXT1'),
('Name1', 'URL1', 'TEXT2')
)
# Make Hashtable K=Name;V={Name, Link, Url, Text}
$AObjects = #{}
# Fill Hashtable from A1
#($a1) |
ForEach-Object {
$name = $_[0]
$AObjects[$name] = #{
Name = $name
Link = $_[1]
Url = $_[2]
Text = ''
}
}
# Fill Hashtable from A2
#($a2) |
ForEach-Object {
$name = $_[0]
$url = $_[1]
if (-not $AObjects.ContainsKey($name)) {
Write-Warning "A2 has [$name] that was not listed in A1!"
$AObjects[$name] = #{Name = $name; Url = $url; Link = ''; Text = ''}
} elseif($AObjects[$name].Url -ne $url) {
Write-Warning "For $($name), A1's URL and A2's URL differ! Using A1's"
}
$AObjects[$name].Text = $_[2]
}
# Convert Hashtable to array of objects with props. Select is required for better output.
$AObjects = #($AObjects.Values) | % { [PSCustomObject]$_ } | Select-Object #('Name', 'Link', 'URL', 'Text')
# Convert array of objects with props back to array
$result = $AObjects |
ForEach-Object {
if ([string]::IsNullOrWhiteSpace($_.Link)) { # Didn't get your logic swapping elements in example
return #(,#($_.Name, $_.Url, $_.Link, $_.Text))
} else {
return #(,#($_.Name, $_.Link, $_.Url, $_.Text))
}
}
#Print
$result | % { "[$($_ -join ',')]" }
# [Name3,Link3,URL3,]
# [Name1,Link1,URL1,TEXT2]
# [Name2,Link2,URL2,TEXT1]
# [Name4,URL4,,TEXT4]
I am trying to use $a variable in this script for working with intermediate steps so that I don't have to use $array[$array.Count-1] repeatedly. Similarly for $prop as well . However, values are being overwritten by last value in loop.
$guests = Import-Csv -Path C:\Users\shant_000\Desktop\UploadGuest_test.csv
$output = gc '.\Sample Json.json' | ConvertFrom-Json
$array = New-Object System.Collections.ArrayList;
foreach ($g in $guests) {
$array.Add($output);
$a = $array[$array.Count-1];
$a.Username = $g.'EmailAddress';
$a.DisplayName = $g.'FirstName' + ' ' + $g.'LastName';
$a.Password = $g.'LastName' + '123';
$a.Email = $g.'EmailAddress';
foreach ($i in $a.ProfileProperties.Count) {
$j = $i - 1;
$prop = $a.ProfileProperties[$j];
if ($prop.PropertyName -eq "FirstName") {
$prop.PropertyValue = $g.'FirstName';
} elseif ($prop.PropertyName -eq "LastName") {
$prop.PropertyValue = $g.'LastName';
}
$a.ProfileProperties[$j] = $prop;
}
$array[$array.Count-1] = $a;
}
$array;
All array elements are referencing one actual variable: $output.
Create an entirely new object each time by repeating JSON-parsing:
$jsontext = gc '.\Sample Json.json'
..........
foreach ($g in $guests) {
$a = $jsontext | ConvertFrom-Json
# process $a
# ............
$array.Add($a) >$null
}
In case the JSON file is very big and you change only a few parts of it you can use a faster cloning technique on the changed parts (and their entire parent chain) via .PSObject.Copy():
foreach ($g in $guests) {
$a = $output.PSObject.Copy()
# ............
$a.ProfileProperties = $a.ProfileProperties.PSObject.Copy()
# ............
foreach ($i in $a.ProfileProperties.Count) {
# ............
$prop = $a.ProfileProperties[$j].PSObject.Copy();
# ............
}
$array.Add($a) >$null
}
As others have pointed out, appending $object appends a references to the same single object, so you keep changing the values for all elements in the list. Unfortunately the approach #wOxxOm suggested (which I thought would work at first too) doesn't work if your JSON datastructure has nested objects, because Copy() only clones the topmost object while the nested objects remain references to their original.
Demonstration:
PS C:\> $o = '{"foo":{"bar":42},"baz":23}' | ConvertFrom-Json
PS C:\> $o | Format-Custom *
class PSCustomObject
{
foo =
class PSCustomObject
{
bar = 42
}
baz = 23
}
PS C:\> $o1 = $o
PS C:\> $o2 = $o.PSObject.Copy()
If you change the nested property bar on both $o1 and $o2 it has on both objects the value that was last set to any of them:
PS C:\> $o1.foo.bar = 23
PS C:\> $o2.foo.bar = 24
PS C:\> $o1.foo.bar
24
PS C:\> $o2.foo.bar
24
Only if you change a property of the topmost object you'll get a difference between $o1 and $o2:
PS C:\> $o1.baz = 5
PS C:\> $o.baz
5
PS C:\> $o1.baz
5
PS C:\> $o2.baz
23
While you could do a deep copy it's not as simple and straightforward as one would like to think. Usually it takes less effort (and simpler code) to just create the object multiple times as #PetSerAl suggested in the comments to your question.
I'd also recommend to avoid appending to an array (or arraylist) in a loop. You can simply echo your objects inside the loop and collect the entire output as a list/array by assigning the loop to a variable:
$json = Get-Content '.\Sample Json.json' -Raw
$array = foreach ($g in $guests) {
$a = $json | ConvertFrom-Json # create new object
$a.Username = $g.'EmailAddress'
...
$a # echo object, so it can be collected in $array
}
Use Get-Content -Raw on PowerShell v3 and newer (or Get-Content | Out-String on earlier versions) to avoid issues with multiline JSON data in the JSON file.
My cmdlet get-objects returns an array of MyObject with public properties:
public class MyObject{
public string testString = "test";
}
I want users without programming skills to be able to modify public properties (like testString in this example) from all objects of the array.
Then feed the modified array to my second cmdlet which saves the object to the database.
That means the syntax of the "editing code" must be as simple as possible.
It should look somewhat like this:
> get-objects | foreach{$_.testString = "newValue"} | set-objects
I know that this is not possible, because $_ just returns a copy of the element from the array.
So you'd need to acces the elements by index in a loop and then modify the property.This gets really quickly really complicated for people that are not familiar with programming.
Is there any "user-friendly" built-in way of doing this? It shouldn't be more "complex" than a simple foreach {property = value}
I know that this is not possible, because $_ just returns a copy of the element from the array (https://social.technet.microsoft.com/forums/scriptcenter/en-US/a0a92149-d257-4751-8c2c-4c1622e78aa2/powershell-modifying-array-elements)
I think you're mis-intepreting the answer in that thread.
$_ is indeed a local copy of the value returned by whatever enumerator you're currently iterating over - but you can still return your modified copy of that value (as pointed out in the comments):
Get-Objects | ForEach-Object {
# modify the current item
$_.propertyname = "value"
# drop the modified object back into the pipeline
$_
} | Set-Objects
In (allegedly impossible) situations where you need to modify a stored array of objects, you can use the same technique to overwrite the array with the new values:
PS C:\> $myArray = 1,2,3,4,5
PS C:\> $myArray = $myArray |ForEach-Object {
>>> $_ *= 10
>>> $_
>>>}
>>>
PS C:\> $myArray
10
20
30
40
50
That means the syntax of the "editing code" must be as simple as possible.
Thankfully, PowerShell is very powerful in terms of introspection. You could implement a wrapper function that adds the $_; statement to the end of the loop body, in case the user forgets:
function Add-PsItem
{
[CmdletBinding()]
param(
[Parameter(Mandatory,ValueFromPipeline,ValueFromRemainingArguments)]
[psobject[]]$InputObject,
[Parameter(Mandatory)]
[scriptblock]$Process
)
begin {
$InputArray = #()
# fetch the last statement in the scriptblock
$EndBlock = $Process.Ast.EndBlock
$LastStatement = $EndBlock.Statements[-1].Extent.Text.Trim()
# check if the last statement is `$_`
if($LastStatement -ne '$_'){
# if not, add it
$Process = [scriptblock]::Create('{0};$_' -f $Process.ToString())
}
}
process {
# collect all the input
$InputArray += $InputObject
}
end {
# pipe input to foreach-object with the new scriptblock
$InputArray | ForEach-Object -Process $Process
}
}
Now the users can do:
Get-Objects | Add-PsItem {$_.testString = "newValue"} | Set-Objects
The ValueFromRemainingArguments attribute also lets users supply input as unbounded parameter values:
PS C:\> Add-PsItem { $_ *= 10 } 1 2 3
10
20
30
This might be helpful if the user is not used to working with the pipeline
Here's a more general approach, arguably easier to understand, and less fragile:
# $dataSource would be get-object in the OP
# $dataUpdater is the script the user supplies to modify properties
# $dataSink would be set-object in the OP
function Update-Data {
param(
[scriptblock] $dataSource,
[scriptblock] $dataUpdater,
[scriptblock] $dataSink
)
& $dataSource |
% {
$updaterOutput = & $dataUpdater
# This "if" allows $dataUpdater to create an entirely new object, or
# modify the properties of an existing object
if ($updaterOutput -eq $null) {
$_
} else {
$updaterOutput
}
} |
% $dataSink
}
Here are a couple of examples of use. The first example isn't applicable to the OP, but it's being used to create a data set that is applicable (a set of objects with properties).
# Use updata-data to create a set of data with properties
#
$theDataSource = #() # will be filled in by first update-data
update-data {
# data source
0..4
} {
# data updater: creates a new object with properties
New-Object psobject |
# add-member uses hash table created on the fly to add properties
# to a psobject
add-member -passthru -NotePropertyMembers #{
room = #('living','dining','kitchen','bed')[$_];
size = #(320, 200, 250, 424 )[$_]}
} {
# data sink
$global:theDataSource += $_
}
$theDataSource | ft -AutoSize
# Now use updata-data to modify properties in data set
# this $dataUpdater updates the 'size' property
#
$theDataSink = #()
update-data { $theDataSource } { $_.size *= 2} { $global:theDataSink += $_}
$theDataSink | ft -AutoSize
And then the output:
room size
---- ----
living 320
dining 200
kitchen 250
bed 424
room size
---- ----
living 640
dining 400
kitchen 500
bed 848
As described above update-data relies on a "streaming" data source and sink. There is no notion of whether the first or fifteenth element is being modified. Or if the data source uses a key (rather than an index) to access each element, the data sink wouldn't have access to the key. To handle this case a "context" (for example an index or a key) could be passed through the pipeline along with the data item. The $dataUpdater wouldn't (necessarily) need to see the context. Here's a revised version with this concept added:
# $dataSource and $dataSink scripts need to be changed to output/input an
# object that contains both the object to modify, as well as the context.
# To keep it simple, $dataSource will output an array with two elements:
# the value and the context. And $dataSink will accept an array (via $_)
# containing the value and the context.
function Update-Data {
param(
[scriptblock] $dataSource,
[scriptblock] $dataUpdater,
[scriptblock] $dataSink
)
% $dataSource |
% {
$saved_ = $_
# Set $_ to the data object
$_ = $_[0]
$updaterOutput = & $dataUpdater
if ($updaterOutput -eq $null) { $updaterOutput = $_}
$_ = $updaterOutput, $saved_[1]
} |
% $dataSink
}
I got a script that creates two arrays (each has 1 column and variable number of lines). I want to format these two arrays and e-mail it to an Outlook account. Code and sample data below.
$Values2 = #(Get-Content *\IdealOutput.csv)
$OutputLookUp2 = #()
foreach ($Value in $Values2) {
$OutputLookUp2 += $Excel.WorksheetFunction.VLookup($Value,$range4,3,$false)
}
$Excel.Workbooks.Close()
$Excel.Quit()
$EmailFrom = "sample#sample.com"
$EmailTo = "sample#sample.com"
$EmailBody = "$Values2 $OutputLookup2"
$EmailSubject = "Test"
$Username = "sample"
$Password = "sample"
$Message = New-Object Net.Mail.MailMessage `
($EmailFrom, $EmailTo, $EmailSubject, $EmailBody)
$SMTPClient = New-Object Net.Mail.SmtpClient `
("smtp.outlook.com", portnumber) #Port can be changed
$SMTPClient.EnableSsl = $true
$SMTPClient.Credentials = New-Object System.Net.NetworkCredential `
($Username, $Password);
$SMTPClient.Send($Message)
Both $OutputLookUp2 and $Values2 are one column with variable number of lines.
Example:
$Outputlookup2 =
X1
X2
$Values2 =
Y1
Y2
I would like the output to the body of the e-mail to be:
X1 Y1
X2 Y2
And I would like to avoid HTML as it will be sent via text as well.
Assuming my interpretation is correct this seems simple enough. For every $Values2, which is just a line from a text file, find its similar value in the open spreadsheet. You are have the loop that you need. Problem is you are building the item lists independent of each other.
$Values2 = #(Get-Content *\IdealOutput.csv)
$OutputLookUp2 = #()
foreach ($Value in $Values2){
$OutputLookUp2 += "$Value $($Excel.WorksheetFunction.VLookup($Value,$range4,3,$false))"
}
Now $OutputLookUp2 should contain your expected output in array form.
If the array does not work you could also just declare it as a string and the add newlines as you are building it. You will notice the "`r`n" at the end of the string.
$Values2 = #(Get-Content *\IdealOutput.csv)
$OutputLookUp2 = ""
foreach ($Value in $Values2){
$OutputLookUp2 += "$Value $($Excel.WorksheetFunction.VLookup($Value,$range4,3,$false))`r`n"
}
In both example you can just flip the order of the $value and the lookup easy. If you need a header you can add that when you declare $OutputLookUp2.
There is always room for improvement
If you want to take this a little further in the direction that Ansgar Wiechers was eluding to...
$OutputLookUp2 = Get-Content *\IdealOutput.csv | ForEach-Object{
"$_ $($Excel.WorksheetFunction.VLookup($_,$range4,3,$false))"
} | Out-String
and thank you in advance for taking a look.
I am having an issue in a script I wrote in Powershell. The script below is a little sloppy so please forgive me.
Basically, this script takes input from a directory of text files. Each file has a line in it like so, with the following structure:
GlobalPath, AgencyPath,SitePath,SizeofSite (in bytes)
\\servername\shared, \8055\Single\department, \sitename,524835900000
The line in question is:
# Split full path and peak usage
$CalculationBuffer = $DailyBuffer[$k].Split(",")
Which results in the following error:
Method invocation failed because [System.Char] doesn't contain a method named 'Split'.
At D:\script.ps1:387 char:52
+ $CalculationBuffer = $DailyBuffer[$k].Split <<<< (",")
+ CategoryInfo : InvalidOperation: (Split:String) [], RuntimeException
+ FullyQualifiedErrorId : MethodNotFound
So my question: Is the array casted incorrectly? Since it is reporting [System.Char] instead of [System.String]?
If the file I am inputting has two lines, it does not result in this error. If the file has only one line, it gets casted as [System.Char] instead.
:Full Script:
# Monthly Output File
[string]$monthoutfile = $ProgPath + "Billing\" + $monthdate + "\_Master_" + $monthdate + ".log"
[string]$currentmonth = $ProgPath + "Billing\" + $monthdate + "\"
# Define what type of files to look for
$files = gci $currentmonth | Where {$_.extension -eq ".log"}
# Create a datastore\dictionary for this month
$MonthDataDictionary = New-Object 'System.Collections.Generic.Dictionary[string,long]'
$MonthAvgDictionary = New-Object 'System.Collections.Generic.Dictionary[string,long]'
# Arrays
$DailyBuffer = #()
$CalculationBuffer = #()
$TempArray = #()
# Counters\Integers
[int]$Linesinday = 1
[int]$DayCounter = 1
[int]$LineCounter = 0
# Strings
[string]$DailyPath = ""
[string]$Outline = ""
# Longs
[long]$DailyPeak = 0
[long]$Value = 0
##########################################################################
# Begin Loop
# Write once...
#$CalcBuffer += "\"
foreach ($file in $files)
{
# First get content from text file and store in buffer
$DailyBuffer = Get-Content $file.pspath
# Determine how many lines are in the file, call function
$Linesinday = linecount $file.pspath
for ($k = 0; $k -lt $Linesinday; $k++ )
{
# Split full path and peak usage
$CalculationBuffer = $DailyBuffer[$k].Split(",")
# Store site path
$DailyPath = $CalculationBuffer[0] + $CalculationBuffer[1] + $CalculationBuffer[2]
# Store peak usage
$DailyPeak = $CalculationBuffer[3]
# Write to dictionary under conditions
# Check if current path is stored or "Site".
# If NOT .ContainsKey($DailyPath)
if (!($MonthDataDictionary.ContainsKey($DailyPath))) {
# Add Key
$MonthDataDictionary.Add($DailyPath, $DailyPeak)
# If it does contain a value
} elseif ($MonthDataDictionary.ContainsKey($DailyPath)) {
# Add the value to the current value for averaging
$MonthDataDictionary.Item($DailyPath) += $DailyPeak
}
}
# Accumulator
$DayCounter ++
}
# Now that each file is tallied up, run an average calculation
$MonthDataDictionary.getenumerator() | Foreach-Object -process {
$Value = $_.Value / $DayCounter
$MonthAvgDictionary.Add($_.Key, $Value)
}
# Debug:
# Write-Host the values
$MonthAvgDictionary
# Output the "Average Peak" values to a file
$MonthAvgDictionary.getenumerator() | Foreach-Object -process {
# Construct output line
$OutLine = $_.Key + "," + $_.Value
$OutLine >> $MonthOutFile
}
This is a known pitfall in Powershell. Just wrap the Get-Content in an array "expression" #() :
$DailyBuffer = #(Get-Content $file.pspath)
I think the problem you're having is that get-content doesn't return an array of strings (lines), but a single string. Thus, when you look at $dailybuffer[k], you're looking at the kth character of the string, not the kth line.
I had the same problem and only the line just like below solved this problem. For your problem it will be:
[string[]] $DailyBuffer = Get-Content $file.pspath