I'm completely lost in a mind loop at the moment:
I wanted to solve the problem of passing an array to a function by value instead of reference and found this solution:
Is Arraylist passed to functions by reference in PowerShell.
The .Clone()-method works as expected:
function testlocal {
param ([collections.arraylist]$local)
$local = $local.Clone()
$local[0] = 10
return $local
}
$local = [collections.arraylist](1,2,3)
'Testing function arraylist'
$copyOfLocal = testlocal $local
$copyOfLocal
'Testing local arraylist'
$local
Output:
Testing function arraylist
10
2
3
Testing local arraylist
1
2
3
But now I need to process the array's elements in a foreach-loop. What happens then is that the array does not get modified by the foreach-loop (???). I am at a loss to understand this, despite a lot of research. Could you please explain to me what is happening behind the scenes and how I can avoid this?
I need to modify a copy of the original array within a function's foreach-loop. In my real script, the array consists of custom PSObjects, but the behavior is the same.
function testlocal {
param ([collections.arraylist]$local)
$local = $local.Clone()
$local[0] = 10
foreach ($item in $local) {
$item = 100
}
return $local
}
$local = [collections.arraylist](1,2,3)
'Testing function arraylist'
$copyOfLocal = testlocal $local
$copyOfLocal
'Testing local arraylist'
$local
Output is not changed by the foreach-loop:
Testing function arraylist
10
2
3
Testing local arraylist
1
2
3
UPDATE 2016-12-14
The tip with the for-loop works, but it turns out when using objects, the whole cloning-thing falls apart again:
function testlocal {
param ([collections.arraylist]$local)
$local = $local.Clone()
for($i = 0; $i -lt $local.Count; $i++){
$local[$i].Hostname = "newname"
}
return $local
}
$target1 = New-Object -TypeName PSObject
$target1 | Add-Member -MemberType NoteProperty -Name "Hostname" -Value "host1"
$target2 = New-Object -TypeName PSObject
$target2 | Add-Member -MemberType NoteProperty -Name "Hostname" -Value "host2"
$local = [collections.arraylist]($target1,$target2)
'Testing function arraylist'
$copyOfLocal = testlocal $local
$copyOfLocal | ft
'Testing local arraylist'
$local | ft
Output:
Testing function arraylist
Hostname
--------
newname
newname
Testing local arraylist
Hostname
--------
newname
newname
Suddenly I am back to passing by reference again. This is driving me mad!
Please help!
When you enumerate the array by calling foreach, you are getting copies of the content and any changes will get discarded. As Mathias Jessen mentioned you can use a for loop to make changes to the arraylist.
EDIT 2016-12-16
Okay, I looked into it and the arraylist clone method does not work like you and I thought it does. According to MSDN, it returns a shallow copy, i.e. it "copies only the elements of the collection, whether they are reference types or value types, but it does not copy the objects that the references refer to". You and I where both assuming that clone would produce what is called a "deep copy".
In your original example using the basic datatype Int32, deep and shallow copy are the same thing, while in your updated example using PSObjects, they are not and only the references are copied. A method to deep clone objects is found in this thread.
I did not find documentation on this, but it seems as if foreach would do shallow copies also:
$object1 = New-Object -TypeName PSObject -Property #{Hostname="host1"}
$object2 = New-Object -TypeName PSObject -Property #{Hostname="host2"}
$array_of_objects = [collections.arraylist]($object1,$object2)
# Looping through basic values, call by value
foreach ($string in $array_of_objects.Hostname)
{
$string = "newname"
}
$array_of_objects.Hostname
# Looping through objects, call by reference
foreach ($object in $array_of_objects)
{
$object.Hostname = "newname"
}
$array_of_objects.Hostname
Hope this helps.
Related
I've been trying to write a utility function that will return initialized [System.Data.DataTable] object data type.
I am aware of the annoying PowerShell functions return behavior where it tries to "unroll" everything into a [System.Array] return type.
Previously I have always managed to get around that problem.
Usually using the "comma trick" to return your own array #(,$result) aways worked - however this time this does not seem to make a difference and I've run out of options...
Another trick I normally use is the $null assignments within the Process block (see my code bellow) - this way I fool the pipeline there is nothing to "unroll" on the output...
I am not taking impossible for an answer, so far based on my experience nothing is impossible in PowerShell :)
Here is my code:
function Get-SourceDataTable
{
[OutputType([System.Data.DataTable])]
[cmdletbinding()]
param(
[parameter(Mandatory=$true, Position=0)]
[System.Data.SqlClient.SqlBulkCopy] $Destination,
[parameter(Mandatory=$true, Position=1)]
[System.Collections.Specialized.OrderedDictionary] $ColumnDefinition,
[parameter(Mandatory=$false, Position=2)]
[int]$ColumnStartIndex = 0
)
BEGIN{
$datatable = New-Object System.Data.DataTable
$colIndex = $ColumnStartIndex
}
PROCESS{
$ColumnDefinition.Keys |
foreach {
$null = $datatable.Columns.Add($_, $ColumnDefinition[$_]) # define each column name and data type
$null = $Destination.ColumnMappings.Add($_, $colIndex) # map column to destination table
$colIndex++
}
}
END{
return ,$datatable
}
}
I hope someone can get this code working...
Rather than return use Write-Output -NoEnumerate. For example:
function New-DataTable {
$datatable = New-Object System.Data.DataTable
$null = $datatable.Columns.Add("x",[int])
$null = $datatable.Columns.Add("y",[int])
$null = $datatable.Rows.Add(#(1,2))
$null = $dataTable.Rows.Add(#(3,4))
Write-Output -NoEnumerate $datatable
}
New-DataTable | Get-Member
Note however that if you just type New-DataTable, it might look like enumberated rows, but Get-Member tells you the actual type returned.
I got the function from the question to return DataTable type when I used LoadWithPartialName to load the assembly containing the type and pipe it out with Out-Null.
Don't ask my why, but feel free to comment if you know the reason.
The working function code is below. Note the return statement is not necessary I only use it to improve code readability:
function Get-SourceDataTable
{
[OutputType([System.Data.DataTable])]
[cmdletbinding()]
param(
[parameter(Mandatory=$true, Position=0)]
[System.Data.SqlClient.SqlBulkCopy] $Destination,
[parameter(Mandatory=$true, Position=1)]
[System.Collections.Specialized.OrderedDictionary] $ColumnDefinition,
[parameter(Mandatory=$false, Position=2)]
[int]$ColumnStartIndex = 0
)
BEGIN{
[System.Reflection.Assembly]::LoadWithPartialName("System.Data") | Out-Null
$datatable = New-Object System.Data.DataTable
$colIndex = $ColumnStartIndex
}
PROCESS{
$ColumnDefinition.Keys |
foreach {
$null = $datatable.Columns.Add($_, $ColumnDefinition[$_]) # define each column name and data type
$null = $Destination.ColumnMappings.Add($_, $colIndex) # map column to destination table
$colIndex++
}
}
END{
return ,$datatable
}
}
To summarize all known possible solutions to the problem of forcing PowerShell function to return specific data type:
use $null assignments
use comma to return an array ,$variable
use LoadWithPartialName("Assembly.Name") | Out-Null
use Write-Output -NoEnumerate $variable to return the type - credit goes to Burt_Harris
Finally, after the input from Burt_Harris (THANKS Burt!) the final working version of the function from this question is this:
function Get-SourceDataTable
{
[OutputType([System.Data.DataTable])]
[cmdletbinding()]
param(
[parameter(Mandatory=$true, Position=0)]
[System.Data.SqlClient.SqlBulkCopy] $Destination,
[parameter(Mandatory=$true, Position=1)]
[System.Collections.Specialized.OrderedDictionary] $ColumnDefinition,
[parameter(Mandatory=$false, Position=2)]
[int]$ColumnStartIndex = 0
)
BEGIN{
#[System.Reflection.Assembly]::LoadWithPartialName("System.Data") | Out-Null
$datatable = New-Object System.Data.DataTable
$colIndex = $ColumnStartIndex
}
PROCESS{
$ColumnDefinition.Keys |
foreach {
$null = $datatable.Columns.Add($_, $ColumnDefinition[$_]) # define each column name and data type
$null = $Destination.ColumnMappings.Add($_, $colIndex) # map column to destination table
$colIndex++
}
}
END{
#return ,$datatable
Write-Output -NoEnumerate $datatable
}
}
I have a function that replaces PackageID in a SCCM task sequence, I would like to capture all those package IDs into a variable, so I would be able to create a report based on that.
The problem is that I already have a foreach loop doing the work, and I can't figure out how to not overwrite the values.
$Driver.PackageID comes from a foreach loop based on $Drivers, which contains
If I run the code I get this as I have Write-Output defined:
Updated code:
function Set-Drivers{
foreach ($Driver in $Drivers) {
Write-Output "Driver Name: $($Driver.Name)"
Write-Output "DriverPackageID: $($Driver.PackageID)"
}
}
$array = #()
$array = Set-Drivers
$hash = [ordered]#{
'DriverName' = $Driver.Name
'DriverID' = $Driver.PackageID
}
$array += New-Object -Typename PSObject -Property $hash
Can someone explain, why I only get the first result in my $array? I can see the values are being overwritten if I run it in debug mode.
Your code is not iterating over the results, but instead only using one of them. This what you intended.
$array = $drivers | foreach {
[ordered]#{
DriverName = $_.Name
DriverID = $_.PackageID
}
}
Your function doesn't return anything. It only writes lines to the console. Then after the function is finished, you create a single object and add that to your array.
Try something like
function Set-Drivers{
$result = foreach ($Driver in $Drivers) {
[PsCustomObject]#{
'DriverName' = $Driver.Name
'DriverID' = $Driver.PackageID
}
}
# output the result
# the comma wraps the result in a single element array, even if it has only one element.
# PowerShell 'flattens' that upon return from the function, leaving the actual resulting array.
,$result
}
$array = Set-Drivers
# show what you've got
$array
I feel like my Delphi background is destroying my ability to figure this out. I'm trying to create an empty array (no data, just structure) in Powershell where each item has two properties. The end result would look something like this:
$WIP[0].signature = 'data'
$WIP[0].PID = 'data'
# other fake stuff in between
Write-host "Item 43 has signature: " $WIP[43].signature
For some reason, I'm roadblocking on every attempt to create what should be simple to do. Thoughts?
Update to answer questions
I know some people do similar to the following, but this isn't as flexible as I'd like:
$array = #()
$object = New-Object -TypeName PSObject
$object | Add-Member -Name 'Name' -MemberType Noteproperty -Value 'Joe'
$object | Add-Member -Name 'Age' -MemberType Noteproperty -Value 32
$object | Add-Member -Name 'Info' -MemberType Noteproperty -Value 'something about him'
$array += $object
This requires the values to be present for all three members when creating each $object. I was thinking the init would look more along the lines of (pseudocode):
$MyRecord = {
Signature as string
PID as integer
}
$RecArray = array of $MyRecord
That's notably a bad mashup of Delphi and Powershell. But would create a fully structured array, addressable as noted up top.
A PSv5+ solution that uses a PS class and a generic list ([System.Collections.Generic.List[]]) to store the instances (loosely speaking, an array that can grow efficiently).
# Your custom type.
class MyRecord {
[string] $Signature
[int] $PID
}
# If you want a list that can grow efficiently,
# use [System.Collections.Generic.List[]]
$RecList = [System.Collections.Generic.List[MyRecord]]::new()
# Add a new instance...
$RecList.Add([MyRecord]::new())
# ... and initialize it.
$RecList[0].Signature = 'sig1'
$RecList[0].Pid = 666
# ... or initialize it first, and then add it.
# Note the use of a cast from a hashtable with the property values.
$newRec = [MyRecord] #{ Signature = 'sig2'; PID = 667}
$RecList.Add($newRec)
# Output the list
$RecList
The above yields:
Signature PID
--------- ---
sig1 666
sig2 667
As for removing objects from the list:
To remove by index, use .RemoveAt(); an out-of-range index throws an error:
$RecList.RemoveAt(1)
To remove by object already stored in the list, use .Remove().
Note that the [bool] return value indicates whether the value was
actually removed (if the object wasn't in the list, the operation is
a no-op and $False is returned)
$actuallyRemoved = $RecList.Remove($newRec)
For details, see the docs.
You want to create a custom object.
You create an object that has all the properties you need. Then you create a collection, and you stuff instances of your new object into the collection. Here's an example:
$WIP = #()
$o = New-Object –TypeName PSObject
Add-Member -InputObject $o –MemberType NoteProperty –Name signature –Value 'foo'
Add-Member -InputObject $o –MemberType NoteProperty –Name data –Value 'bar'
$WIP += $o
$WIP[0].signature
$WIP[0].data
You'd need to execute the New-Object and Add-Member statements for each object you're creating.
So here's working example of how You can get something like this working:
$list=#()
1..100|foreach-object{
$obj=""|select signature,pid
$obj.signature="$_ signature"
$obj.pid="$_ PID"
$list+=$obj
}
With the object created this way - You can do $list[43].signature and it does work.
What exactly do you mean by "Dynamic"?
$array = #(
# Some type of loop to create multiple items foreach/for/while/dowhile
foreach ($item in $collection) {
New-Object psobject -Property #{
Signature = 'data'
PID = 'data'
}
}
)
Or you can manually add objects like so
$array = #()
# Later in code
$array += New-object psobject #{
Signature = 'data'
PID = 'data'
}
Then you can access each item like so:
$array[1].Signature
$array[1].PID
There is no real difference between this an what you have already been shown but I think this gives you what you are asking for (even though it is not the powershelly way to do things).
$object = "New-Object PSCustomObject -Property #{'Name' = ''; 'Age' = [int]}"
$array = 1..100 | %{Invoke-Expression $object}
$array[0].Name = 'Joe'
$array[0].Age = 12
$array[0]
You can use a hashtable with indices as keys, and your hashtable as values. It's pretty easy to work with.
$WIP = #{
0 = #{
signature = 'signature 0'
PID = 'PID 0'
}
1 = #{
signature = 'signature 1'
PID = 'PID 1'
}
}
You can add any index you want.
$WIP[12] = #{
signature = "signature 12"
PID = "PID 12"
}
$WIP[12].PID
# PID 12
You can initialize both, any, or none.
$WIP[76] = #{
signature = "signature 76"
}
$WIP[76].signature
# signature 76
$WIP[76].PID
# $null
Count gives you number of "active" elements.
$WIP.Count
# 4
I am trying to use $a variable in this script for working with intermediate steps so that I don't have to use $array[$array.Count-1] repeatedly. Similarly for $prop as well . However, values are being overwritten by last value in loop.
$guests = Import-Csv -Path C:\Users\shant_000\Desktop\UploadGuest_test.csv
$output = gc '.\Sample Json.json' | ConvertFrom-Json
$array = New-Object System.Collections.ArrayList;
foreach ($g in $guests) {
$array.Add($output);
$a = $array[$array.Count-1];
$a.Username = $g.'EmailAddress';
$a.DisplayName = $g.'FirstName' + ' ' + $g.'LastName';
$a.Password = $g.'LastName' + '123';
$a.Email = $g.'EmailAddress';
foreach ($i in $a.ProfileProperties.Count) {
$j = $i - 1;
$prop = $a.ProfileProperties[$j];
if ($prop.PropertyName -eq "FirstName") {
$prop.PropertyValue = $g.'FirstName';
} elseif ($prop.PropertyName -eq "LastName") {
$prop.PropertyValue = $g.'LastName';
}
$a.ProfileProperties[$j] = $prop;
}
$array[$array.Count-1] = $a;
}
$array;
All array elements are referencing one actual variable: $output.
Create an entirely new object each time by repeating JSON-parsing:
$jsontext = gc '.\Sample Json.json'
..........
foreach ($g in $guests) {
$a = $jsontext | ConvertFrom-Json
# process $a
# ............
$array.Add($a) >$null
}
In case the JSON file is very big and you change only a few parts of it you can use a faster cloning technique on the changed parts (and their entire parent chain) via .PSObject.Copy():
foreach ($g in $guests) {
$a = $output.PSObject.Copy()
# ............
$a.ProfileProperties = $a.ProfileProperties.PSObject.Copy()
# ............
foreach ($i in $a.ProfileProperties.Count) {
# ............
$prop = $a.ProfileProperties[$j].PSObject.Copy();
# ............
}
$array.Add($a) >$null
}
As others have pointed out, appending $object appends a references to the same single object, so you keep changing the values for all elements in the list. Unfortunately the approach #wOxxOm suggested (which I thought would work at first too) doesn't work if your JSON datastructure has nested objects, because Copy() only clones the topmost object while the nested objects remain references to their original.
Demonstration:
PS C:\> $o = '{"foo":{"bar":42},"baz":23}' | ConvertFrom-Json
PS C:\> $o | Format-Custom *
class PSCustomObject
{
foo =
class PSCustomObject
{
bar = 42
}
baz = 23
}
PS C:\> $o1 = $o
PS C:\> $o2 = $o.PSObject.Copy()
If you change the nested property bar on both $o1 and $o2 it has on both objects the value that was last set to any of them:
PS C:\> $o1.foo.bar = 23
PS C:\> $o2.foo.bar = 24
PS C:\> $o1.foo.bar
24
PS C:\> $o2.foo.bar
24
Only if you change a property of the topmost object you'll get a difference between $o1 and $o2:
PS C:\> $o1.baz = 5
PS C:\> $o.baz
5
PS C:\> $o1.baz
5
PS C:\> $o2.baz
23
While you could do a deep copy it's not as simple and straightforward as one would like to think. Usually it takes less effort (and simpler code) to just create the object multiple times as #PetSerAl suggested in the comments to your question.
I'd also recommend to avoid appending to an array (or arraylist) in a loop. You can simply echo your objects inside the loop and collect the entire output as a list/array by assigning the loop to a variable:
$json = Get-Content '.\Sample Json.json' -Raw
$array = foreach ($g in $guests) {
$a = $json | ConvertFrom-Json # create new object
$a.Username = $g.'EmailAddress'
...
$a # echo object, so it can be collected in $array
}
Use Get-Content -Raw on PowerShell v3 and newer (or Get-Content | Out-String on earlier versions) to avoid issues with multiline JSON data in the JSON file.
My cmdlet get-objects returns an array of MyObject with public properties:
public class MyObject{
public string testString = "test";
}
I want users without programming skills to be able to modify public properties (like testString in this example) from all objects of the array.
Then feed the modified array to my second cmdlet which saves the object to the database.
That means the syntax of the "editing code" must be as simple as possible.
It should look somewhat like this:
> get-objects | foreach{$_.testString = "newValue"} | set-objects
I know that this is not possible, because $_ just returns a copy of the element from the array.
So you'd need to acces the elements by index in a loop and then modify the property.This gets really quickly really complicated for people that are not familiar with programming.
Is there any "user-friendly" built-in way of doing this? It shouldn't be more "complex" than a simple foreach {property = value}
I know that this is not possible, because $_ just returns a copy of the element from the array (https://social.technet.microsoft.com/forums/scriptcenter/en-US/a0a92149-d257-4751-8c2c-4c1622e78aa2/powershell-modifying-array-elements)
I think you're mis-intepreting the answer in that thread.
$_ is indeed a local copy of the value returned by whatever enumerator you're currently iterating over - but you can still return your modified copy of that value (as pointed out in the comments):
Get-Objects | ForEach-Object {
# modify the current item
$_.propertyname = "value"
# drop the modified object back into the pipeline
$_
} | Set-Objects
In (allegedly impossible) situations where you need to modify a stored array of objects, you can use the same technique to overwrite the array with the new values:
PS C:\> $myArray = 1,2,3,4,5
PS C:\> $myArray = $myArray |ForEach-Object {
>>> $_ *= 10
>>> $_
>>>}
>>>
PS C:\> $myArray
10
20
30
40
50
That means the syntax of the "editing code" must be as simple as possible.
Thankfully, PowerShell is very powerful in terms of introspection. You could implement a wrapper function that adds the $_; statement to the end of the loop body, in case the user forgets:
function Add-PsItem
{
[CmdletBinding()]
param(
[Parameter(Mandatory,ValueFromPipeline,ValueFromRemainingArguments)]
[psobject[]]$InputObject,
[Parameter(Mandatory)]
[scriptblock]$Process
)
begin {
$InputArray = #()
# fetch the last statement in the scriptblock
$EndBlock = $Process.Ast.EndBlock
$LastStatement = $EndBlock.Statements[-1].Extent.Text.Trim()
# check if the last statement is `$_`
if($LastStatement -ne '$_'){
# if not, add it
$Process = [scriptblock]::Create('{0};$_' -f $Process.ToString())
}
}
process {
# collect all the input
$InputArray += $InputObject
}
end {
# pipe input to foreach-object with the new scriptblock
$InputArray | ForEach-Object -Process $Process
}
}
Now the users can do:
Get-Objects | Add-PsItem {$_.testString = "newValue"} | Set-Objects
The ValueFromRemainingArguments attribute also lets users supply input as unbounded parameter values:
PS C:\> Add-PsItem { $_ *= 10 } 1 2 3
10
20
30
This might be helpful if the user is not used to working with the pipeline
Here's a more general approach, arguably easier to understand, and less fragile:
# $dataSource would be get-object in the OP
# $dataUpdater is the script the user supplies to modify properties
# $dataSink would be set-object in the OP
function Update-Data {
param(
[scriptblock] $dataSource,
[scriptblock] $dataUpdater,
[scriptblock] $dataSink
)
& $dataSource |
% {
$updaterOutput = & $dataUpdater
# This "if" allows $dataUpdater to create an entirely new object, or
# modify the properties of an existing object
if ($updaterOutput -eq $null) {
$_
} else {
$updaterOutput
}
} |
% $dataSink
}
Here are a couple of examples of use. The first example isn't applicable to the OP, but it's being used to create a data set that is applicable (a set of objects with properties).
# Use updata-data to create a set of data with properties
#
$theDataSource = #() # will be filled in by first update-data
update-data {
# data source
0..4
} {
# data updater: creates a new object with properties
New-Object psobject |
# add-member uses hash table created on the fly to add properties
# to a psobject
add-member -passthru -NotePropertyMembers #{
room = #('living','dining','kitchen','bed')[$_];
size = #(320, 200, 250, 424 )[$_]}
} {
# data sink
$global:theDataSource += $_
}
$theDataSource | ft -AutoSize
# Now use updata-data to modify properties in data set
# this $dataUpdater updates the 'size' property
#
$theDataSink = #()
update-data { $theDataSource } { $_.size *= 2} { $global:theDataSink += $_}
$theDataSink | ft -AutoSize
And then the output:
room size
---- ----
living 320
dining 200
kitchen 250
bed 424
room size
---- ----
living 640
dining 400
kitchen 500
bed 848
As described above update-data relies on a "streaming" data source and sink. There is no notion of whether the first or fifteenth element is being modified. Or if the data source uses a key (rather than an index) to access each element, the data sink wouldn't have access to the key. To handle this case a "context" (for example an index or a key) could be passed through the pipeline along with the data item. The $dataUpdater wouldn't (necessarily) need to see the context. Here's a revised version with this concept added:
# $dataSource and $dataSink scripts need to be changed to output/input an
# object that contains both the object to modify, as well as the context.
# To keep it simple, $dataSource will output an array with two elements:
# the value and the context. And $dataSink will accept an array (via $_)
# containing the value and the context.
function Update-Data {
param(
[scriptblock] $dataSource,
[scriptblock] $dataUpdater,
[scriptblock] $dataSink
)
% $dataSource |
% {
$saved_ = $_
# Set $_ to the data object
$_ = $_[0]
$updaterOutput = & $dataUpdater
if ($updaterOutput -eq $null) { $updaterOutput = $_}
$_ = $updaterOutput, $saved_[1]
} |
% $dataSink
}